“Calhoun 


Institutional Archive of the Naval Postgraduate School 





Calhoun: The NPS Institutional Archive 
DSpace Repository 


Theses and Dissertations 1. Thesis and Dissertation Collection, all items 


1982 


A distributed routing protocol for a packet 
radio network 


Heritsch, Robert R. 


Monterey, California. Naval Postgraduate School 
http://ndl.handle.net/10945/20133 


This publication is a work of the U.S. Government as defined in Title 17, United 
States Code, Section 101. Copyright protection is not available for this work in the 
United States. 


Downloaded from NPS Archive: Calhoun 


: Calhoun is the Naval Postgraduate School's public access digital repository for 
/ (8 D U DLEY research materials and institutional publications created by the NPS community. 
«ist : Calhoun is named for Professor of Mathematics Guy K. Calhoun, NPS's first 


NY KNOX appointed — and published — scholarly author. 


LIBRARY Dudley Knox Library / Naval Postgraduate School 
411 Dyer Road / 1 University Circle 
Monterey, California USA 93943 





http://www.nps.edu/library 





é 
i] 
r) 
' 
"' 
i] 
i] 
' 
| 
' 
1 Cia 
' 
| 
| 
4 
i 








te SU 
i ; ane my ee ees ' 
i " J if 7 7 - : Ns ‘th rt ul ii 4" va 
' a5 ab a A” \" 
' j i | : | i i , 4 
X ee a yo COCs 
A i; 4 TPA , 1 f ‘ 
Ff . 
vi Vv “gu i 1. iV 
a 
i 
7 a) Par a ' be 
| Ps ' a 
| \ ; 
ie | @ | ‘| ‘Py. 
4 i, i iy ‘ b : 1. : ] i 
af Swe i ft. A ( ‘ Sih uv ae fH 
L te : H a ' 
yo | ee y 
vi ip ey ni ee ae ei Jae 
; \ ua) ee : a, : s iyor AS 
' aS ‘ 7 “a i 
+ Pte yo 
i} Pak a 
: | G 
4 
i Ly ' i ar i 
i’ i ‘ 
‘ toe 
t 2 
| af 1 ' me 
ft ! ‘ ; ‘ 
i ii ia ' r 
i i ar on : 
i ( , - ' LF i rm i : 
Ae! a +) ; 
i} A ; a es 
’ ‘ 7 ‘ ie ws tat 
\ ‘ ; A Hy u t 
= oe ' nie : 
‘ on 4 a : oe suheahal 
os pan, i 
: A i a} 
' ij et fy Vert 
‘ an revel 
' : ‘ I 7 . . ' reli 
' 5 ' ‘ ' a st 
: ' om | ) » 1) i 1 . 
‘ yh 3 VW 
0 i, - | - :F 
i} i i a 
‘ | 
ion oe phy “ 
a 
i 
F | : - 
, i} 
) 
‘ $y. 
' tf : 
1 
' 
i ‘ 
WH) 
' ' 
| 
is 
a) 
; | 
\ nye y 
rl ‘ 
' 1 
i} 
‘ - 1 
a abs oe 
tT) : - ti Lee 
« : ° i 
° i yah 
; tt eae ir a 
Pa iY 
a , oP Wee ae 
if r Ff a i tb vv 
sy ¥) 1 
ve 7 ~ 
i 7 i: 
* if i 
% ir. on spor 
af : 
7 : a? 
: i ‘ ‘ 
i 4 Aa : 
os ea 
oo 4 \ av 
i a é 
1 a ' wee eee RY 
; ‘ 1 U0) 0 
> AK 
i, 4 
{ ‘ iy 
mo 
: i L 
i) a7 * 
vs i a! we 
“se : . . oF we ou 
| - 1, is 5 i Vv Vv woe 
1 ie a I 
| al ; Y 7 Lt i oe ; a 
. i Hy ee 
i : i ah 
i - 
mae Mt 
ar : ; 
1 on ae A yh a 
af =" 4 | { ' ; 
vin L) , Lal Pa A . i A 
i , 
\ ; , ive ‘ 
| 1 ' 7 aed 
' i i} ' 7 i 
1 | owe 
i i a 
: Al LP yA 
i ieate cen coy 
i 1F Bes 1 
| | \ , mo" if rT) 
| ( A ie aan a , ; ie ey} yi 
' . ALL TT Aa 
i i, ' | Ay 7. ap 1 the i bl 














NAVAL POSTGRADUATE SCHOOL 


Monterey, California 





2 th oo beh 


A DISTRIBUTED ROUTING PROTOCOL 
FOR A PACKET RADIO NETWORK 


by 
Robert Heritsch 


March 1982 


Thesis Advisor: J... Wozencraft 





Approved for public release, distribution unlimited 





sal 
SECURITY CLASSIFICATION OF THIS PAGE (When Dote Entered) 


REPORT DOCUMENTATION PAGE 


2. GOVT ACCESSION NO. 






READ INSTRUCTIONS 
BEFORE COMPLETING FORM 


RECIPIENT'S CATALOG NUMBER 













3. 













S. TYPE OF REPORT & PERIOD COVERED 
Master's Thesis 
March 1982 


6. PERFORMING ORG. REPORT NUMBER 


4. TITLE fand Subtitie) 




















MEbiictmt outed Routing Protocol for a 
Packet Radio Network 


7. AUTHOR(#) 8. CONTRACT OR GRANT NUMBER(2) 


Robert R. 





Heritsch 





10. PROGRAM ELEMENT. PROJECT, TASK 


9. PERFORMING ORGANIZATION NAME AND AOORESS AREA &@ WORK UNIT NUMBERS 








Naval Postgraduate School 
Monterey, California 93940 










12. REPORT OATE 
March 1982 


13. NUMBER OF PAGES 


LOVE 


18. SECURITY CLASS. (of tate report) 


CONTROLLING OF FICE NAME AND ADORESS 





11. 
Naval Postgraduate School 
Monterey, California 39490 
















AGENCY NAME & ADORESSE(I! different trom Cantroliing Ollice) 





DECLASSIFICATION/ DOWNGRADING 
SCHEDULE 


16. DISTRIBUTION STATEMENT (of thle Report) 










meproved for public release, distribution unlimited 


BUTION STATEMENT (ol the ebetract entered in Block 20, ii different from Report) 





17. OmsTRri 


18. SUPPLEMENTARY NOTES 


19. KEY WORDS (Cantinue on reverse side if necessary and identify wy slock number) 














Packet Radio, Rouwmeunag  Peorcoco., Distributed Routing Algorithm 








Digital Communications, Network Management 








20. ABSTRACT (Continue an severee side i{ necessary and identity by afeck number) 
meme Radio 15 a Gigital communications concept which offers the 
user the capability to pass voice and other data ina radio 
Network which may link high power computers with small mobile 
radios containing microprocessors. The techinique of routing 
G@mgatal tratfic from source to destination depends on the 
Operational requirements of the network. Most routing concepts 
today centralize network control (in varying degrees) for normal 





DD ~oM™ 1473 EDITION OF | Nov 68 13 OBSOLETE 


1 JAN 73 
° e } ne a A 
san ERE? ee SECURITY CLASSIFICATION OF THIS PAGE (Phen Deore Entered) 





a 


DD Form. 1473 
Ne 
S/N 0102-914-G6601 


rr er ea Te 
Secumty CL ASSiFIC avian OF Tih PaGdsWren Nore Satreced 





operations. This thesis describes a concept for completely 


decentralized control of a packet radio network. The basic 
Bre@uocel is relatively simple and robust, but suffers from the 
usual build-up of overhead traffic with network size. Another 


related routing protocol is proposed which, under certain 
Operational situations, reduces routing traffic and memory 
requirements compared to the basic algorithm. A concept for 
use of alternate links in the event of a broken link is also 
suggested. 


2 


a ee 
SZOCUAITY CLABBIFIC ATION OF Tuis PadEsWren Sate Errered 


. pA eS EE OO a EE NAP 
a re a 


Se EEE eee ee eee 





Approved for public release; distribution unlinite 


A Distributed Routing Protocol for a Packet Radio Network 
by 


Robert Heritsch 
Major, United States Army 
B.S., University of Wisconsin-Milwaukee, 1969 


Submitted in partial fulfillment of the 


requirements for the degree of 
MASTER OF SCIENCES IN ELECTRICAL cNGINEERING 


Crom the 


NAVAL POSTGRADUATE SCHCOL 
March 1982 


oy P. 





ABSTRACT 


maw Ines nyv 


UDLEY KN! 


Packet Radio is a digital communications concept which 


offers the user the capability to 


eveat. Lc 


computers with small mobile 
microprocessors. 
from source to destination depends 
requirements of the network. Most routing 
centraiize network control -(in 


operations. 


in a radio network which 


The teachnigue of 


This thesis describes 


pass voice and other data 


may link high power 
radios containing 


EOUtENgeergital trarfic 


Varying degrees) for normal 


a concept for completely 


decentralized control of a packet radio network. The basic 


protocol is relatively simple and 


robust, but suffers fron 


the usual build-up of overhead traffic with neétwork size. 


Another 
Sertain 
memory 

concept 


menk is 


related routing protocol is proposed which, under 


Operational situations, reduces routing traffic and 


requirements compared to the basic algorithn. A 


for use ct alternate links 


also suggested. 


in the event of a broken 


On the operational 


concepts today 


= SCHOOL 


a 





iS LEO CONTENTS 


se MNPOUUET BON. « « «© «© © © © © © ° 
A. THe PACKET KRADTO CONCEPT ...- .. 

Be ROUTING. % 6s «6 «© «© «2 © « s So 

1. Centralized and Backbone Systems . 

2. Completely Decentralized Routing . 

Pol. MNO RA COUREENG . »© © © © © 0« © » « ee 
Ae NOUGS es «6 « 6 « 6 «© «© ¢ © « Se 

Sie [SIGN Gs cena se 5 6 6 6 tle! seus 

Ci Come ves «6 « 6 © «© «© « « ss 

D. NEPMNORK DieneeeeS . 6 « «© «6 «© « »« ee 


- 


a ROoUEIne OF Jser Trazfic .. 


Pee GLOKS s+ « «= + «© « « 


Soa Newest. eo 6 Goes) elle 


uw. Network Maintenance Traffic 


Laie Dewees Or wemomamsaolhD PROTOCOL . . . 


aXe 


NODcwLOmreIne PHRO@LOCOL. s . s « «+ 
1. Establishing and Monitoring 
Zu MAeGxree NaekmowLeagement . . .« 
NETWORK MANAGEMENT PROTOCOL... 
Tc (0G 2.2) a 


Zo Soleo SeakGOwn  < . « «© «© * e 


10 


11 


12 


15 


15 


16 


18 


19 


19 


21 


G2 


24 


25 


JA 


Zo 


30 


33 





A DISTRIBUTED NETWORK MANAGZMENT FROTOCOL CONCEPT 


Ae 


see 


Upeme oprevECh PROTOCOL . « « «© « « « 


le 


he 


3. 


WomiGe@membicertC . ss « « © «© « « 
Daocameibincdihe le os .« . 6 0 « © e« « e 


Integrated Management Traffic . 


Sey Geer SON WION 6 ww el lt 


Doe EES. O) |S) a a ne a a ae a 


Eve 


EXPANDING TO RELATED GROUPS AND FAMILIES 


1. 


Ze 


TcmecisaGMGLOUD ~. «si <6 +6 « « « 
NGtNeEeTOS 6 6 6 « ¢ «© + © @ “s-« 
Messe Sercuns «eicMis @ ss « «@ “s 
QumOIpOae= mmeSSagC ens « 6 2 « « 
D. Bpexen= Path Message ...«-. 
(UNGee | es «ess 6 « «© 6 « -« 
Normal Operations without New or 
TERI SMCMETSTE Ss clo ie 6 s+ « «© « « 
Introducing New and Broken Links 


Alternate Link - an Interin Fix 


MOnmecr Vek rIOnS «<6 «6 0.6 © -« 


Bete sonetes Of GEOUpPanG . . . -« 


SIMULATION @ e e e @ e @ e e@ e e e e e @ 


Ae 


Is) 


SIMULATING A OSER AND MEASURING EFFECTIVENESS 


PeGivabiemenG GCMEME . 2. << s « »« » « » 


34 


Bes. 


40 


44 


47 


3 0 


54 


35 


55 


56 


64 


ue 


75 


12 


78 


a1 


ee. 


95 





Ge ARRAYS AND PSMPORARY ENTITIcS ..-. 

D. SELECTION OF TEST PARAMETERS . « « » 

VI. RESULTS, CONCLUSIONS AND RECOMMENDATIONS 
ae. ReswbomAwD OSSERVAMEONS . « + © « « 

1. peBias2c Geoup Tests . . .« . « « « 
eoeciteelyy7GEOUr TestS . . ml. 6 «+ 

5. CENGUUSEGHNS (es Sew Re ew lw 

Cs RECOMMENDATIONS FOR FURTHER STUDY . 

SE ROOMEKTADGs 6 <« se © © «© © © « « © 6 © © «@ « 
Perro rxX B . 6 © © © « © © © © © © © © © 8 ew 
PCCD Pc 4. 6 5 4) 6s s+ 6 ss « © © «© «© © © » 
APPENDIX D es eR oes Bsleco cs es <6 «© « 
Mr OE es ic elle «© “« « « © «© « © © © © » 
PPOs Ar NOG 25 «6 « ¢ © 6 * «¢ #© © © © © # « 


Gemerial DISTREBUTION LIST . . « « «© © «© «© «© «@ 


UPS) 


123 


125 


Ta0 


133 


138 


142 





ACKNOWLEDGEMENT 


Mie soppOLtunity <O work Closely with Prof. Wozencraft, 
whe patiently placed many of these concepts in my mind, was 
truly the high point of my studies at NPS. The constant 
Support and understanding of ny wife Sandy, who placed most 
of these concepts in print, has helped make this entire 


period of study richly rewarding and enjoyable. 





I. INTRODUCTION 


ie THE PACKET RADIO CONCEPT 

Packet Radio technology extends the appiication of 
packet switching intc the mobile radio environment. Ee 
offers a convenient and efficient way to communicate among a 
large number of mobile users. This is particularly 
important in a tactical environment where rapid deployment 
and mobility are required. 

Users in a packet radio network essentially share common 
radio channels. Use of these channels 1S (to varying 
degrees) controlled by microprocesscrs in the user's radio 
in a manner which is transparent to the user. Packet Radio 
moma digital communications concept which in principle can 
accommodate voice as well as digital data traffic provided 
that adequate traffic capacity is availiable. The use in 
packet radio of spread spectrum communications is 
Particularly attractive to military applications because of 
potential capabilities for a low probability of intercept 
(LPI) and excellent antijamming (AJ) characteristics. 

Although the military 1S pressing development in packet 
radio technology, it is also, ina sense, part of the 


natural evolution of the computer age. Almost all computer 





networks are bound to the cables which connect the 
computers. Yet as computers get smaller and potentially 
more mobile, the need for wireless links become more 
important. 

Two packet radic network testbeds are currently in 
Operation. The Bay Area PRNET (packet radio network) in Sana 
Francisco has been operational since 1976, and is the 
primary site for development and evaluation of network 
protocols and apolication’ “Sencepts. The Army Data 
Distribution System (ADDS) testbed PRNET at Ft. Bragg, North 
Carolina, became operational in 1979 with the objectives of 
providing potential users of packet radio technology with 
exposure to the technology early in its development, giving 
timely feedback to developers and offering the users an 
Opportunity to experimentally determine the impact on 
tactical doctrine of mobile access to computer-based command 


ana control. 


B. ROUTING 

The goal of a properly operating packet radio network is 
tO route packets (groups of bits) thru a series of radios 
from the sender to the receiver in an efficient manner. 


Enroute, the packet is automatically processed and passed on 
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by aseries of tadios in a manner transparent to those 
users. Through multiplexing or other concepts, a radio may 
provide input/output service to 1ts user and also forward 
other traffic simutaneously. Because of the limited power 
of mobile radios, a transmitter may often not have a direct 
link with the ultimate receiver. Ina military application, 
low power transmissions may also enhance survivability. 
Therefore the routing from every potential message source to 
every potential destinaticn requires the application ofa 
network-wide intelligence to determine the most efficient 
links along which to forward the message. There are two 
Main, different approaches to routing algorithms for solving 
this problem. 
1. Centralized and Backbone Systems 

Both the Bay Area PRNET and the Ft. Bragg PRNET use 
network components called stations to manage the routing in 
different porticns of the network. fo ene. Fe. Bragg 
network, each station is a (DEC) PDP 11/40. The purpose or 
Mie station is to monitor the celative activity level in 
each radio under its jurisdiction, and to aid in the routing 
of traffic that passes thru, originates or terminates in its 


portion of the network. 
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There may be many wierons in a Network, each 
controlling a certain number of user radios. Together taney 
provide the network-wide intelligence which maintains and 
implements the dynamic routing scenerio. Network control is 
centralized in the stations. This scheme is both practical 
and efficient. However ina military sense, stations-are a 
vulnerability since only a few of them control the operation 
of all the radios in the network. 

Use of a backbone network offers similar advantages 
and shortcomings. A backbone is a network superimposed over 
a common user network which improves the efficiency of the 
network by providing high volune, high speed and/or long 
distance trunks. Traffic from the common user 1S placed on 
and taken off of the backbone in accordance with a routing 
process such as the station concept mentioned above. Once 
again, the vulnerability of the network is directly related 
to the vulnerability of the backbone. 

2. Complstely Decentralized Routing 

A completely decentralized network does not have 
Stations or a backbone. Conceivably, every user has a 
Meeket cCadlio containing a microprocessor which is no 


different than any other communicaticns/processing component 
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(other packet radios) in the network. Depending on the 
topographical situation, there may also be unattended packet 
radios in the network. These radios do not have users which 
use the radio as a terminal into and out of the network. 
They are usually placed in positions in the network to 
provide additional communication paths or links increasing 
“he number of routing alternatives. However these 
unattended radios function essentially the same as a 
terminal user's radio insofar as message processing is 
concerned. hieescenere ¥ords, in a decentralized network, 
every cadio is the same and there is no centralized of 
semi-centralized component controlling how the network 
operates. It is the collection cf packet radios themselves 
which must combine their processing capabilities to create 
the network-wide intelligence needed to build, maintain and 
implement an efficient routing scneme for user traffic. The 
advantage is a reduction in the vulnerabilities inherent in 
any system which tends to centralize its COnt sou 
Sapeadllities. The disadvantages are increased complexity, 
increased overhead traffic (which represents competition 
Mine user traffic for a finite channel capacity), and 


possibly a reduction in speed. 
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The objective of this study is to present and 
investigate the performance of a Simple algorithm whica 
could be programmed into each packet radio in a completely 
decentralized network. Assuming that each radio in tne 
network has a very limited range compared to the diameter of 
the network, the aigorithm allows each radio to relay 
information about other radios (called nodes from now on) 
throughout the network. The algorithno uses this 
information, as it works its way through the network, to 
create relatively efficient communication paths (links) 
between every pair of radios (nodes) in the network. The 
end result automatically gives users throughout the network 
the appearance of direct access to every other node in the 
network, albeit with some delay. The dynamic routing of 
traffic as it is created and enters the network enabies many 
channels of communications to exist Simutaneously across the 


network. 
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Il. NETWORK MODELING 





Al*hough this study is based on what 1S considered a 
meackical Concept for a wmilitary radio network, the theory 
Can be considered very general in nature. Therefore, the 
network is modeled as a combination of nodes and links 
petween nodes. Furtanernmore, the nodes and links are 
affected dynamically by events such aS routine traffic, the 
gain or loss of a link, and network maintenance ¢rafric. 
This chapter defines the modeling components and functions, 
relates them *o physical components or requirements, and 


makes some aSSumptions. 


ae 6 NODES 
tie che Goedel, nodes represent recelver-transmitters. 
Nodes also contain processors. It 1S convenient to picture 


many functions in each node performed by parallel processors 
So Bia.e all unrelated processing can be performed 
Simucaneously. Conversely, only those operations which must 
be performed in a sequence with a significant execution time 
meeewSoupject to conflicts and queuing delays. 


All nodes in toe network have exactly che same 


Capabilitiss. However, depending on its processiz 
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process a given message 


mSsceuctilons, each node may 

ger Peer ont loye For example, one node may be a terminal for a 
Therefore, this node may accept routine 

determine which 
Sat 


specific user. 
a certain list of addresses, 
user, deliver 


its assigned 
the appropriate 


m@eatlic for 
traffic 1s addressed to 
traffic, and retransmit the remainder to 

Another node may be solely a transmitter which 


addresses. 
only relays routine traffic and does not serve as a terminal 


for a user. 
When one nodé can pass trarfic directly to another node, 
Chemevrst node. 


is considered a neighbor to 
Although every node in 


she other node 


These nodes are connected by a link. 
Our network can cecntact every other node, each node has only 


a limited list of neighbors which may vary with time. 
are in direct contact 
when one or 


LINKS 
whenever two nodes 


a 
A link is considered broken 
Or receive 


A link exists 
Or; 


with each other. 
the capability to transmit 
a link implies two-way 


both nodes lose 

eeem, the other node. Therefore, 
communications between specific node pairs. Of course the 
actual method of communications in a radio network is 
through antenna transmissions. These may be either 
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directional or omni-directional antennas. Ande Of COUrSS, 
these transmissicns could potentially be received by many 
nodes other than a particular partner ina node pair. 
Conceptually, «his can be accommodated by assuming <hat all 
traffic/packets contain tne address of the iatended 
receiving node for a given link. Then any node which 
receives ‘traffic not addressed to it Simply ignores the 
messade. 

Another aore sophisticated concept has a link 
representing a unigue center frequency which one node uses 
#o transmit to another. In creating the link, the two nodes 
determine which frequency bands are mutually available, and 
then 2ach selects an available transmission frequency to 
communicate with the other node. Now, when either aode 
wishes +9 transmit to the other, i¢+ uses its selected 
Erequency band. Conceptually, only one node within range of 
gauge ven transmitter will accept traffic ina particular 
frequency band. In this manner more than one link toa 
Single node may be operating simutaneously. Other more 
familiar techniques such as Code Division Multiplexing could 


also be used to establish a link, 
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Ge CHANNEL VALUE 

Assuming that a network consisting of many links has 
been established, one needs an efficient way to use this 
network. Clearly, an unacceptable technique would be to 
retransmit every message on every link to ensure that the 
addressee receives the message. Although 1t may ensure chat 
a single message gets to its destination, it represents work 
for ever node in the network. Assuming that dirrerent 
messages could be initiated by many nodes in the network, 
peas cnat much cf this trafric couid be present in the 
network at the same time. The inefficiencies of 
broadcasting quickly lead to saturating nodes or links in 
the network, since nodes indiscriminantly relay everything 
they hear. Smart nodes should be able to do much better. 


What is needed is a way of selecting one link over 


another link. Once that decision is nade, ToariLiee ror a4 
given destination uses only the best path, or optimun 
series of links, rrom the source of a message to its 


destination. One way to quantify the connection between two 
nodes is to assign a weight or cost value to each link or 
Channel in the network. Then, summing the costs for a given 


path between ‘two nodes, one can asSign a value <9 every 
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possible path, and thereby (theoretically) pick the lowest 
cost path between a source and destination. 

There may be many ways to assign channel values. One 
practical technigue would be to count the backlog of traffic 
(or packets) waiting to use a particular link. This queue 
or delay represents a portion of the total time it takes for 
a message to reach its destination. Normally it is desired 
that traffic move through the network as quickly as 
possible. This 1S particularly important if the network is 
to accommodate real-time speech. Therefore, a channel vaiue 
which reflects net transmission time is useful. This is the 


technique used in this study. 


dD. NETWORK DYNAMICS 

Nodes and links represent the static network structure. 
But a practical network oust accommodate changes which aay 
be represented as the creation or destruction of nodes or 
links. Furthermore, there must be a concept for passing 
network maintenance information and, most importantly, user 
mealtic. 


1. Routine or Jser Traffic 





A network exists to pass routine traffic. Geadi rac 


could be either inter-active voice (characterized by 
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real-time conversations), or data (characterized by one-way 
transmissions assembled or stored at the receiving end for 
later review). 

ipaudegqatdbeenetWonk, Dorh voice and data traffic 
mre cransmitted in the form of digital packets. FOL voice 
traffic, the most important thing is that packets arrive at 
pepeelatively uniform rate. Voice packets are created by 
sampling the voice signal. The number of voice bits 
reguired per unit time is a function of the encoding 
technique and the desired quality of the received signal. 
Any additional bits are unnecessary and therefore waste 
channel capacity. Fewer bits, in the form of delayed or 
lost voice packets, may degrade the reception. Note taat 
Sree a VOice packet is delayed one inter-packet period,it is 
Momelonger useful. 

For data traffic, it is not necessary to have a 
Smooth £lOow of traffic. Bursty traffic is quite acceptaoble. 
The important thing is that after the message is divided 
into packets for transmission from the source node, ail 
these packets are recovered and reassembled properly at tae 


destination node to recreate the original message. 


20 





The ability for data packets to move satisfactorily 
in a bursty manner allows them to complement the frigid 
pehedule of voice packets. A concept tor the integration of 
voice and data traffic is discussed in more detail in 
G@Hapter III. 

2. Broken Links 

As defined earlier, a link lmplies the capability 
for two-way communications between two nodes. A broken link 
is recognized ina node when it is discovered that this 
two-way capability no longer exists. Depending on the 
situation, as explained in Chapter III, the two nodes on 
each side ofa link may realize a link is broken at 
different times. 

In modeling a network, a broken link may be used to 
represent various events. If a nodé is lost, it could be 
reflected as a brceken link between the lost node and each of 
its neighbors. If the transmission path between neighbors 
is interrupted, this can be represented as a loss of a 
Single link between the two nodes. If links are broken ina 
BeectcCilar pattern, it may indicate that a varticular node 


is moving away from its neighbors. 
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3. New Links 

AS opposed to a broken link, anew link is created 
when the nodes on each end establish communication with each 
other. This will typically require some interaction between 
the two nodes. 

New links would be created when an inactive node 
becomes active, when a moving node moves into range of other 
nodes, or when other conditions change to enable two-way 
communications between two nodes where conditions previously 
prevented this link. 

It 1S apparent that a nétwork can be dynamically 
modeled by allowing links to be broken or created to 
represent physical activities such as changing signal paths, 
nodes entering and leaving the network (being turned on or 
off), node movements and other situations. 


4. Network Maintenance Traffic 





If the nodes in a network are to be as organized and 
rescurceful as described above, then they must be programmed 


to communicate with each other, pasSing information related 


| 


momeneir activity and capabilities. In a network with fuily 
distributed control, the objective is to achieve efficient 


network-wide communication under the constraint that each 


aie 





node can only transmit and receive directly with a limited 
number of neighbors. tvemeutouwue no Central contre] facalicy 
+o route and monitor traffic between non-adjacent acdes. 
Every user in the network must be able to reach 
every other user in the network in a manner wWhico is 
transparent to all users, even in a dynamic environment 
where links are created or broken randomly. Therefore, over 
and above user traffic, nodes must pass nétwork maintenance 
cima tt iC. This traffic should be transparent to the user. 
This means that the nodes measure or sense thelr operational 
Status and are programmed to automaticaily E epee 
information to their neighbors. Neighbors process’ the 
information and may then automatically relay the processed 
information to selected neighbors until every node requiring 
*he information eventually receives it. A program or 
algorithm that generates and processes networx maintenance 
meaatic is commoniy called a protocol. At a @insanun, to 
model a practical network, network maintenance traffic nust 
accommodate new links, broken links, and changes in channel 
values which may represent more efficient ways of routing 
Memeane tratfic through the network. The concept of 
protocols for distributed networks is discussed in much 


Smeeacce detail in Chapter III. 
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Tis  Dbevebo OF DESTARIBUTED PROTOCOL 





A distributed protocol ina packet radio network is 
defined as algorithms which are executed independently in 
each node +o process both network naintenance and routine 
mea tific. The effect should be the overall efficient use of 
network resources, approaching the efficiency of a centrally 
@emtrOlled network. 

A particular protocol may be based on many design 
considerations. Of course the designer must consider the 
capabilities of available or proposed equipment and the 
characteristics of the operating medium. But within these 


constraints, the designer may be free to trade off such 


hinds aS Simplicity and robustness ror speed and 
sophistication. And of course these qualities ara not 
mutually exclusive. Therefore, the examples of distributed 


protocols in the literature vary from rather limited, siaple 
ones such as Yen's algorithm (Ref. 1], to more 
sophisticated and complicated algerithms such as 
mecalil's [{Ref. 2}. 

It may be helpful to break down network operations 
oerformed by each node into three groups or levels of 


Pmotocol. ia this way activities can be isolated, 





controlled and analyzed in a modular fashion while assuming 


the remainder of the node's functions are unaffected and 
operating as expected. This study assumes three levels of 
Beogecol. The first is node to node protocol, the,second is 
network management protocol and the third is user service 
Beonocol. Concepts and examples cf node to node protocol 
and user service protocol are discussed in some detail in 
this chapter. Network management protocol is mentioned only 
DeeeEly in this chapter. However, a detailed concept and 
2xample is daveloped and analyzed in the remainder of this 


Seeuliady . 


A. NODE TO NODE PROTOCOL 

There are séveral activities required in an active 
network which basically involve only two nodes. Perhaps the 
most fundamental interaction 1S recognizing each other. 
This mutual recognition is considered a link. Links exist 
=o vass Prarc lc, which leads to another important 
inter-nodal function, ‘that of the receiving node informing 
she sending node that it has received its message. 

if Ss cedlishing and Monatoring a Link 

A nede to node protocol should provide for 


2stablishing communications between two nodes. Tas could 


ILE, 





be accomplished by each node asynchronously transmitting a 
beacon message on a designated frequency. The beacon 
message would contain the identity of its originator. Any 
other node receiving the beacon message with an adequate S/N 
ratio checks its list of neighbors. If the node addressed 
in the beacon message is not found on the receiving node's 
neighbor list, the receiving node would initiate an 
acknowledgement message addressed to the node which sent the 
beacon message. I= the original rode now receives the 
acknowledgement, it adds the node which sent tae 
acknowledgement message to its neighbor list and sends that 
nod¢ a notice message that two way communications exist. 
Finally, the nede which initially responded to the beacon 
message adds the originator of the beacon messageto its 
MeaGgiubor list and a anéw link is born. 

AS mentioned in Chapter II, the implementation of a 
link may vary by design. tip, cOLeexanplecy = the twoe@way Link 
actually consists of two frequency bands which eénabdie 
Simultaneous transmission between two nodes ona single 
tink, then the interchange of information in establishing 
the link would include the determination of autually 


available frequency bands. ite, Onewecnhe othes @ handy the 
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network used Carrier Sense Multiple Access (CSMA), which was 
the technique actually used in the DARPA packet radio 
testbed which Operated in the San Francisco Bay 
area [Ref. 3], then different information must be passed to 
ee@eablish a link. 


Once established, the status of a link must be 


monitored. The beacon message could also be used for this 
Poneti1on. A node should expect to recéive beacon messages 
from every node on its neighbor list. Therefore, when it 


receives the beacon message there ls no need to fCcreply. 
However, 1% may note the time it received the last beacon 
message from each of its neighbors. Failing to receive a 
beacon message from a neighbor over an established period of 
me would promcrt a node to conclude that it had lost tyo 
Way communications. This may have an impact on many other 
nodes in the network and would therefore initiate a reaction 
by the Network management protocol as discussed in paragraph 
2 below. When a node discovers it has lost a link, the 
corresvonding node on the other end of the link must be 
removed from its list of neighbors. 

There 1S another more immediate way for a node to 


meeacover that it has lost a link. This would occur when a 


ei) 





node attempted to send a packet to a neighboring node but 
does not receive an appropriate acknowledgement for a 
successful transmission. If this is the case, the sending 
node could try to retransmit at least one more time, but 
2ventually it may conclude that the link has been broken. 
Once again this may initiate activity by a higher level 
ee TvOocol. This also demonstrates how one node may discover 
that a link has been broken before it is discovered by its 
corresponding neighbor. In any packet radio concept, the 
establishment and monitoring of links is a fundamental 
activity that can be delegated to a low level protocol. 
2. ~ Packet Acknowledgement 

Another node to node function is the acknowledgement 
by the receiving node to the sending node that a packet has 
peen successfully transmitted across a link. Under any 
practical operating concept for a packet radio aetwork, 
there are significant opportunities for a node to improperly 
recelve a packet. A few of these situations include 
multipath interference, latentional e) 3 unintentional 
jamming, fading cr improper synchrcnization. A conservative 
design consideraticn would preclude a transmitting node froa 
purging a transmitted packet from its memory until it has 


received acknowledgement that the packet has been receivad. 
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In the ALOHA net, operated by the University of 
Hawaii, this acknowledgement is accomplished as the sending 
node monitors the retransmission of the receiving node. If 


the retransmissicn matches what waS sent, the sending node 


eliminates the packet from its memory. hw ordLGsnoe, -- the 
packet is tresent. This is an adeguate technique for an 
ALOHA~type network. But if a node uses different 
frequencies for each link, 16 Jolie technigue may be 


impractical. 

Another concept 1S to terminate each packet with 
check bits. The number of check bits per packet would be a 
function of the expected probability of error per bit for an 
average link. If ali check bits are properly received, the 
Meeerving node reports its successful reception to <the 
sending node ina brief message. Lack Of Such) atespert 
after an established period of time may prompt a node to 
retransmit a packet. Receipt or an acknowledgement would 
cause a node to eliminate the packet from itS memory, 
Sems.dering it successfully transmitted. Theré are cneck 
bit schemes for fail-safe communications which are not only 
more efficient than a bit-for-bit check, but are also more 


reliable [Ref. 4]. 
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Q@Pher veritication Gencepts May offer stiil other 
advantages. However, because of the inherent potential ana 
S2onificant effects if bit error in radio communications, .it 
is very likely that some technique of ensuring accurate 
packet transmission will be required in any packet radio 


network. 


B. NETWORK MANAGEMENT PROTOCOL 

The primary objective of a communications network is to 
move user traffic from source to destination. A network 
management protocol is intended to organize the network sc 
mao trarfic moves efficiently under alli conditions. 

If one assumes that node to node activities are 
appropriately handled by a lower level protocol as described 
above, then he can treat the loss or addition of another 
ieee dS a2 Foutine event, and process the information as it 
welmua affect the entire network. Therefore, network 
management protocol is not concerned with hew or under what 
Conditions a link is established. Ee OnLy “decs On. the 
information that a link does or does not exist. 


1. Uvdate i 





The fundamental network-wide management operation is 


ehe update. In an operational network, traffic on each link 
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is constantly changing. To efficiently use the network to 
pass traffic between two given nodes, it is desirable to 
find the "Best Path" between the two nodes. Exactly what is 
measured may be a subjective decision. But once made, this 
quantity can be used to compare various alternatives and 
select a best path. Yet the best path can be expected to 
vary with time, for loading on each link of a network may be 
constantly changing. Therefcre best paths must be updated 
periodically to accommodate network dynamics. 

In a distributed control network, each node could 
initiate its own update. The form of this update message 
and exactly how it is processed in the network depends on 
she selected protccol. There is always a design trade-off 
involving the frequency of updates with the corresponding 
generation of ufdate messages (management traffic) versus 
*he effects of cld or outdated pest paths. This tradeoff 
Should not be a casual decision. In a network of n nodes, 
there are at least n(n-1) best paths. With some of the most 
efficient algorithms, it may take at least (n*®*3) node +o 
node messages to complete one neétwork-wide update under the 
worst conditions (see Appendix C). Therfore it is desirable 
*o find an effective update frequency which provides for 


meeeose2c and efficient network traffic flow. 
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In addition to updating existing paths, the updating 
process can serve to introduce new links into the network. 
In some protecols such as Segall's (Kef 2) the arrival of a 
new link has an immediate impact on the network update 
process. As in the case of broken links discussed next, 
Segall immediately initiates a new update message whenever a 
node experiences a change in its link status. This creates 
a Situation where update messages initiated by the same node 
may be negotiating the network at the same tine. Therefore 
there must be provisions to prioritize these messages so 
that the most recent message taxes precedence over the 
outdated messages. thas / "Ys normally accomplished by 
introducing cycle numbers as part of each update message and 
Many other network management messages. The problem with 
cycle numbers is that they can potentially grow larger than 
eae allotted buffer space. Segall places a bound on his 
cycle numbers by using a precedure devised by Finn [BRef. 5]. 

One of the objectives of this study Sco 
investigate a network management protocol that does not 


require cycle numbers. In tnais concept new links are 


ri 
ct 


introduced to the network only during routine updates. 
is assumed that the delay involved may be traded for 


increased simplicity. 
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2. Link Breakdown 





Broken links may have various impacts on a network. 
If a link is under heavy use, a break nay have a serious 
Berect On net traffic flow. Heavy use may also indicate 
that many other nodes rely on this particular link in their 
best paths to other distart nodes. On the other hand, some 
links may serve very few nodes, and in fact be inactive at 
the time a break is discovered. 

The objective of any reaction to a broken link is to 
Minimize its impact on the flow of traffic and other network 
metivity. Ideally, at bapapt should immediately and 
automatically be switched to the next best path. One way to 
incorporate aiternate links under certain circumstances is 
described in Appendix A and is considered along with the 
proposed network management protocol in Chapter IV. 

When it cannot always immediately reroute traffic, 
the network management protocol must take action to stop or 
reduce traffic intended for a broken link, cause the network 
to find new best paths for traffic affected by the break, or 
a combination of both. Finding new best paths is typically 
done during an update operation. Teese bansunet i one of a 


protocol to indicate how an update may be initiated. 
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In some protocols discovery of a break nay initiate 
an update. For example, the discovering node may broadcast 
a special update request message addressed to 606 gach 
destination for which the discovering node had considered 
*he broken link as part of a best path. Other nodes scho 
the request and eventually the destination nodes receive 
their requests ard initiate an update. Pi wesnas. Concept, 
some type of cycle number would be reguired to mediate 
conflicts between new and outdated updates froma single 
destination node which could exist in the network at the 
Same time. 


Miteanative ly, a—orOLocol can be designed to 


routinely issue updates from each nede ata rate that 


ny 


ensures that any previously issued update message fron 
particular node had already passed out of the network, yet 
often enough to tolerate freezing traffic blocked by a 
broken link until the next routine update provides a new 
best path. This is the basis of the network management 


feerecOl proposed in Chapter IV. 


Moose SERVICS PROTOCOL 
Once a network is constructed and operational, the last 


question is how routine user traffic will be packaged and 
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processed by each node in the network. This could be 
considered the User Seérvice Protocol level. In this case 
the designer may assume that lower ievel protocols will work 
independently to do things such as update best paths, react 
to new or broken links, and acknowledge transmissions across 
a link. The User Service Protocol uses selected information 
from lower-level prcetocols to efficiently accomplish its 
primary function of passing user traffic. 

The basic characteristic of a user service protocol ina 
packet radio network is that, like other lower level 
protocols, it should be transparent to the user. Decisions 
such as packet size, content, and processing depend on the 
Capabilities of the selected equipment and the priorities of 
the network designer. In Pie Sees eeta ON Sa paxrtacular 
Megerithm is discussed as an example of atypical user 
eavi.ce protocol. It is presented to iiiustrate one 
possible technique fcr managing routine traffic within the 
framework of a network operating with other lower ievel 
peetocois. 


im  VYorLce Traffic 





Several assumptions must be made in order to gain 


physical appreciation of the requirements of a conceivable 
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nacket radio network. Some of the parameters selected both 
here and in the remainder of this study are based on a 
theoretical packet radio concept propesed for a Marine 
Amphibious Brigade by Bond {Ref. 6], and Lucke (Ref. 7]. 

It is assumed that the network will move both voice 
mmo data traffic. In this section we discuss the 
characteristics and requirements of each type of trafiic. 
AS mentioned in Chapter II, voice must flow at a consistent, 
periodic rate. Data, on the other hand, can move in bursts 
as channel capacity becomes available. 

In a digital network, voice must be converted to a 
Gegatal signal (vocoding). This is done by sampling tne 
analog voice signal and converting each sample to a digital 
value, This produces a voice packet. In a real-tine 
conversation, any delay of more than approximately 0.1 sec 
between speakers becomes noticeable. Therefore a voice 
packet should take no more than 0.1 Sec to move from tae 
source node to the destination node. Assuming that packets 
will be relayed by a maximum of 10 nodes in our theoretical 
network, and further assuming that processing time in each 
node is far more Significant than the propagation time 


between nodes, then the maximum processing delay per node is 
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~i sec 





Delay = = .01 sec/node/packet. 


10 nodes 


Because cf the periodicity requirements of voice, 
there are certain advantages to establishing a virtual link 
between the traffic source and destination. In a packet 
radio network, a virtual link may consist of reserving a 
time slot on each link along the best path from the source 
to destination at the time the virtual link is established. 
Once a virtual link is established, it is used until the 
source node has finished the voice conversation (unless a 
link is broken), regardless of whether or not subsequent 
update operations have found other best paths during the 
course of the conversation. This ensures periodicity in the 
voice «raffic, for each voice packet passes thru the sane 
number of nodes, with the same net precesSing time for a 
given source-desctination pair. 

In a practical network, a link would probably be 
required to accommodate traffic for more than one node ata 
time. As implied in the previcus paragraph, this may be 
accomplished by Transmitting traffic FORA Specie 
destination node in an assigned time slot on each Link. 


This is also called time division multiplexing. The 
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particular slot on each link enroute to a destination is 
determined as the call is being initiated and the virtual 
Mmenx is being built. Once established, this slot will only 
carry voice packets for its assigned destination until the 
virtual link is broken or dismantled. 

It is convenient to define the series of time slots 
Which can each carry a Separate virtual link as a Frame. 
Then in each frame, one slot represents one virtual link to 
a destination. During normal link operation, each frame is 
followed by another frame carrying the next voice packet in 


the assigned slot for each virtual link (see Fidq. 3.1). 
+ <— 1 | Z | 3 | sequence of slots 


sequence 
Ie 


em—{ 1] 2/3] 1] 2] 3] 1] 2]... of 


frame frane frame 


Pig. 3.1. Slot/Frame Concept 


It was estimated above that if a maxinum of 10 nodes 
were used in a virtual link, each node can take up to .01 


Seer co retransmit one voice packet. Poros 25 “LUE Cher 
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assumed that each frame must handle up to 10 virtual links 
(slots), then each slot (which carries one voice packet) can 
be no more than 1 msec because each frame can last no more 
than .01 sec. 


-01 sec/frame 
10 slots/frane 





1 msec/siot. 


Slot Duration = 


It is estimated that good quality digital adaptive 
Delta-mod voice requires a bit rate of 16 x (10%*3) 
bits/sec. In the multiplexing system mentioned above, #2ach 
voice channel has only a 1/i10th duty cycle. Therefore when 
active, a virtual link must pass traffic at a rate of 160 x 
(10**3) bits/sec. 

For this example, 1f the radios in this network 
operat? with a bandwidth of approximately 100MHZ (spread 
Spectrum), a Significant past detection processing gain 


could be obtained 





_ Transmission Bandwidth 1Q0** 8 
Bandwidth of Message 160 x (10% 3) 


Se OS. 
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2. Data TraMiic 


Data traffic would not normally have the stringent 
*iming requirements that voice traffic may require. On the 
@emer hand, within reason, voice traffic could afford to 
randomly lose packets while experiencing a graceful 
degradation in the actual rlow of information, whereas any 
lost data packets represent an absolute loss of information. 
Therefore the network may pass data traffic more slowly, but 
must de so more accurately. 

Because of the periodicity requirement of voice 
eeaffiic, voice packets need to have priority over data 
packets. Under this network concept, data traffic would be 
integrated as a filler in available slots during pauses in 
vOice traffic. The result is bursts of data «raffic, which 
does not lend itself to the virtuai link concept described 
mere voice traffic. In fact it may be Simpler to picture 
each data packet as an individual message containing the 
address of the destination, and being released by the source 
node to find its way to the destination node. One advantage 
of this concept is that if the network updates its best 
paths while a source node is releasing data packets for a 


particular destination, later packets have the advantage of 
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using the updated best paths to their destination. BY 
contrast, in the virtual link concept considered here, once 
a virtual link is established, @rat fates “Confined “to it 
even though better paths may become availabie. 

If data traffic is to be moved on the network 
developed earlier, it must be able to work within the 
frame/slot concept devised for voice traffic. Assuming a 
slot has a duration of 1 msec with a data rate of 160 x 
(10%*3) bits/sec, then each slot contains approximately 150 
Mees Of information. In the virtual link concept this may 
be perfectly acceptable because once the virtual link is 
established, nearly all bits passed on the virtual link are 
ineer traffic. However, if each data packet is to move 
independentiy from the source node to the destination node, 
each packet must contain certain overhead information which 
is commonly lumped together at the beginning of the packet 


in a preamble. 


PREAMBLE YSER DATA 





DESTINATION : MSG.NO. : PACKET NO. : SOURCE : USER ID 


Fig. 3.2. Preamble 
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The preamble in Fig. 3.2 illustrates some of the 
informaticn that might be required in the heading of a data 
packet. If this information were to require a portion of 
the available bits in every slot, it would seriously degrade 
the rate at which user data could pe passed. An alternative 
is to use much larger data pacxets. 

Data packets are typically created as the source 
node divides up a stream of data from a buffer which is 
being fed by a censole, facsinile device, etc. Themeeze of 
the packets is dictated by the user service protocol. 
Therefore the number of data packets needed to carry the 
users entire message is obviously a function of the messages 
size and the size of a data packet. On the receiving end, 
not only must all data packets be received (correctly), but 
it may be required +o sort the packats +o place them in the 
proper order, méaning each packet must be serial numbered. 
Information such as this does not contribute to the net flow 
of user information. Therefore to pass the largest possible 
GFatio of user information to preamble information with the 
Slot technigue, a data packet including preamble should be 


some larger multiple of a voice packet. 
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When data packets are transmitted across a link, the 
sending node reads the preamble of the data packet and 
assigns a slot number on the best path link toward the 
destination node. The sending node then divid2s the data 
packet into sub-packets which are the size of a siot. 
Depending on the standard size of a data packet, the sending 
node sends the remainder of the data packet in the 
appropriate slot in consecutive frames. The receiving node 
is also programmed to accept a standard humber of 
Sub-packets once it has agreed to accept a data packet ina 
Seeei cular slot. In this way only one preamble is sent per 
data packet and the effective ratio of user information 
actually passed could be significantly increased. 

This procedure is essentially ancther version of the 
weeetial link. Depending on the number of sub-packets and 
System pricrities for handling sub-packets, a virtual link 
mor a data packet may vary in size. For exampl2, if nodes 
are programmed to relay sub-packets as soon as they are 
successfully received, several nodes on the best path may be 
relaying portions of a Single data packet at the same time. 
me tact, the destination node may be receiving the first 


suopackets before the last subpackets are transmicced. The 
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@itference is that these virtual links have a fixed finite 
lifespan. They are limited by the amount of time the 
designer wants to make a slot unavailable to vcice traific. 
An extension of the same idea nas two or nore slots in the 
same frame being used to pass sub-packets of the same data 
packet. This provides a more erficient use of a link which 
may have little voice traffic and 1s consistent with the 
Pipesty nature of traffic. 
3. integrated Management Traffic 

With the exception of the preamble, there has been 
no mention of management traffic which is required by node 
to hode and netwerk management protocols. Tyeacdliyy this 
traffic consists of relatively short messages. It is 
conceivable «hat these messages could be tagged on the end 
of user packets piaced in each slot. In this situation it 
would appear to the network that 100 percent of channel 
capacity waS available to user trafic. et See. Ss NOs 
practical, then slots could be used on an as-needed basis to 
pass groups of Management messages. 

There is another aspect of traffic management that 
mayne considered. Once voice traffic is interrupted, it is 


important that the speaker be notified. ThES could be a 
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programmed response to the network's reaction to a broken 
fmk. The cesult would be for the speaker to guit taiking. 
He viaaiyelLOb dara thattic it is practical for the 
source node to release only a limited number of data packets 
into the network and wait for a teceipt acknowledgement fron 
the destination node as data packets arrive. This is cailed 
ieaLew control", This prevents a source node rom loading 
interim nodes with excessive traffic which the network may 
not be able to process because of a lost link to the 
destination. It also allows the source node to selectively 
retransmit packets that were not successfully received and 
erase those that were. Finally, it provides assurance that 


*he data traffic was received. 
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IV. A DISTRIBUTED NETWORK MANAGEMENT PROTOCOL CONCEPT 





This chapter describes a particular concept for a 
distributed control network management protocol. This 
protocol is limited by design to fit into the larger concept 
of independent levels of protocol which handle different 
classes of messages, processed as described in Chapter III. 
Analysis of the pretocol developed here by a computer 


Simulation is discussed in Chapter V. 


MeoCllLING TH& FRAMEWORK 

The following network management protocol is based on 
the assumption that an adequate node to nede protocol is 
performing necessary functions such as periodically testing 
menks, discovering new as well as broken Ilinks, and 
confirming when a packet has been successfully transmitted 
across a link. 

It 1s further assumed that the result of this protocol, 
Which is intended to be a flexible network which can react 
to link changes and find new best paths based on the latest 
Channel values, wili be used by a higher level user service 
mmetocol. This higher protocol could resembie that 


described in Chapter III. But it is not necessary to define 
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a particular user service protocol in order to investigate a 
lower level network management protocol. Therefore the 
remainder of this study will minimize any assumptions about 
the form of higher level protocols which may use the results 


of this network management protocol. 


See OEFINLTIONS 

All the most common components of our network, such as 
nodes and links, have already been mentioned. However it is 
necessary here to ‘further describe certain previously 
defined components, and to present additional components or 
concepts needed to explain the protocol. 

1. The Basic Group 

The Basic Group is what has been derined as the 

MemewOrk up +o this point. Me oasie GEOup 25 a collection of 
nodes, each having a unique identification, each being 
connected to at least one other node in the basic group, and 
each node being considered an equal member of the group (See 
ol By using only links belonging to the basic 
gzoup, it is possibie to send a nessaga from any node in the 
basic group (called the Source) to any other node (called 


the Destination). 
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Fig. 4.1. Example of a Basic Group 


Our network management protocol will initially be 
developed with nothing more than a basic group. Later, a 
version of the protocol involving ‘Related Groups" and 
"Pamilies of Groups" will be introduced. However this will 
have little impact on the basic concept. 

Mi Grades tO Meve user tratiic efficiently, the 
protocol must be able t0 calculate the best route from a 
source to destination node. Tomeado “this: each link is 
assigned a channel value, and these values are summed and 
compared to determine the best path from the source to 
destination node. It is not essential to specify in advance 


the exact physical nature of these channel values, Or 
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distances as they are sometimes called. But whatever 
channel value physically amounts to, it should reflect the 
relative "cost" of sending traffic over a link at the tine 
it is measured. 

A best path implies that, oased on existing links 
and current channei values at the time it was measured, 
there is at least one combination of links whese net channel 
value represents the most efficient path from the source to 
destination. This is frequently considered the amglninun 
delay route. Best paths can become outdated for two 
reasons: ¢ither one of its links is broken making novement 
impossible, or ancther combination of links develops a -lower 
net channel value. 

7 should be noted that each lizk is astwo way 
communications channel, and usually the current channel 


value in one direction has no relationship to the channel 


value in +the other direction. In Fig. 4.2 below, the 
Channel value from nodes A to Bis 1. However the channel 
mane £rom nodes B to A ils 5. This means that for any two 


nodes in a baSic group, the best path from the first node to 
the second is not necessarily the best path from the second 
Mogae to the first. Thus in any basic group of N nodés, 


there are N(N-1) cr approximately N**2 possible best paths. 
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Channel values 


L S 


Fig. 4.2. Channel Values on a Two-way Link 


2. Activities 

In the course of maintaining the network, the 
protocol will cause each node to initiate and participate in 
several management activities. Most have already obp¢en 
mentioned and will only be discussed priefly here. 

The best path update is the fundamental operation of 
mes level of prctocol. As channel values change and best 
paths become outdated, steps must be taken to find the new 
best path. This process is automatically and asynchronously 
initiated by each node, and when it is completed (which may 
Tequire the Origination of several update cycles as 
discussed below), every other node in the basic group knows 
the latest best path to the initiating node. This operation 


is periodically required cf every node in the network. The 


50 





reason why complete updating of the best paths may reguire 
more than one initiation or an update cycle can be seen in a 
simple example. In the network in Fig. 4.3, node Aa sends 
out an update message to nodes 3 and cC. Node C updates its 
channel value toaA from 5 te 4 but still retains its old 
best pato thru node 3 believing i+ has a total channei vaiue 
od Se Finally after node B relays A's update nessage to C, 
node C learns that the actual channel value thru node B to A 
moenOW 7. When A initiates its next update, node C will 


change its best path to node A to be the direct A-C link. 


¢ channel Touiieicomas Cee ase 
Upadaree 
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Fig. 4.3. Update Iterations 


A broken link can be a tratimatic event in th 


aM 


network. Therefore the protocol will react to broken links 


im an attempt to minimize the effect on user traffic flow. 
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Pirst it will attempt to switch all traffic nampered by the 
broken link to an alternate link. An alternate link is 
defined only in respect to individual nodes, and if one is 
available, it may be used by a node if the node is faced 


with an inability to move traffic over a previous best path 


which now contains a broken link. An aiternate link can 
only be considered as a temporary fix. Its oniy guarantee 
[emeiat if used, it will not create a loop situation. A 


loop is defined as a closed path consisting of a series of 
links. Therefore traffic leaving a loop node will 
eventually return to that node. In Fig. 4.4%, node 2 cannot 
consider the link to node 4 as an alternate link if node 4 
Goutes traffic destined for node 1 through node 3. Pals 
creates a loop. However iz node 2 can be assured that node 
Meet not route traffic destined for node 1 on any path 
which eventually moves tnrough node 2, then node 2 caa 
SWitch traffic tc node 4 after a break with confidence that 
it retains a loop-free network. 

Although switching traffic of a best path implies a 
decrease in efficiency, the alternate may be to stop all 
meter ic routed over a broken link. Of course this may be 


even less efficient. But a node faced with the decision may 
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not always have the option of an alternate Link. See 
Daragraph C3 below and Appendix A for a discussion and proof 
Seedh alternate link concept which 1s compatible with the 
network management protocol described in this chapter. 
Clearly, it is required that a node be able to cope 
With a situation where it may lose ail access to one or more 
nodes. Recovery is defined as eventually establishing 


another path to the disconnected trodes. The erficiency of 


a8 





recovery is defined as the speed at which a path is 
reestablished over the new best path. 

To accomplish the above activities, each node in the 
network will create, process and relay nessages from other 
nodes. The processing wiil frequently change components of 
a message that a node receives from a previous node, adding 
informaticn to the message before relaying it. Nodes are 
also selective as to which other nodes it will send or relay 
a message. The net result is to improve network-wide 
operation and efficiency. 

3. Messages 

The network management protocol is required to send 
two types of overhead messages related tec the maintenance 
activities mentioned in the previous paragraph. Sach 
message will have several elements which will be abbreviated 
and represented in a message argument. 


ae Update Message 


The syieol £05 “an update message and its 
components are shown below. The letter 1 identifies the 
last node to relay the update nessage (or U-msg). The 
letter d identifies the originator of the U-msg. Note that 


when the originator first sends the U-msg, l=d. 0D(1) is the 


cummulative channei value on the best path from 1 to d. 





Update Message ==> U(1,d,D(1l)) 


b. Broken Path Message 
The symbol for a broken path message and its 
components are shown below. The argument d represents the 
destination node for which the broken link is blocking 
Meertic, and corresponds to the d in the U-msg. The d in 
the U-msg is the identity of the initiating node, and 
represents the destination to which the best paths created 


feeecnis U-msg will point. The d in the X-msg indicates that 


the best path to dis broken. 


Broken Path Message ==> X(d) 


eee uae CONCEPT 

The objective of this network management protccol is to 
provide a single algorithm that can operate autonomously in 
each nod= of a network to provide completely decentralized 
Mmemewors COntrol, yet provide for efficient traffic touting. 
This algorithm was also chosen for its relative simplicity 
and potential robustness. Its primary departure from most 


Other aigcrithms of this nature is that it attempts to 





accommodate new and broken link events without requiring 
cycle numbers. The algorithm is given in Appendix B. 
1. Normal Operations without New or Broken Links 
It is most convenient initially to study the update 
process while freezing the status of nodes and links. We 
Will also initially assume each node has a best path to 
every other node. As mentioned earlier, the basic group or 
network consist of N nodes. The number of links between 
these nodes will normally exceed the number of nodes. 
Normally, if they are evenly distributed, the more links 
into an average node, the nore robust is the network. 
Popectemerenrly  WSeewa network, tratfic should take 
the best path from the source a5 destination node. To 
identify and use this path, each node along the way must 
know the destination of the traffic, and what neighboring 
node is downstream on ‘the best path tc each destination. 
Downstream will imply movement toward the destination, that 


is, caliaying the traffic to another node with a smaller 


Ccummulative channel value to the destination. Upstream 
implies movement away from che destination, normally 
backwards along the best path. The update message allows 


2ach node +o determine which neighbor is on its best path to 
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every other node in the network. Each node periodically 


initiates a U-msg to all its neighbors. [In Fig. 4.5 node 1 


NK 


7 
sa = 


Pig. 4.5. Initiating an Update 


initiates an update by sending U(1,1,0) tonodes 2 and 4. 
When a node receives a U-msg ititiated by d, it computes the 
cummulative channel value to d thru 1 and compares it to the 
last cummuliative channel value along the node's current best 
Mach to ad. For example, in Fig. 4.5, suppose node 4 had 
previously selected the direct link, with channei value = 5, 
as its best path to node 1. Meanwhile node 2 has aiso 
received a U-msg from node 1, has determined that this is 


its best path tc nede 1 because no other path offers a 





cummulative channel vaiue of 1, and has relayed node 1's 
U-msg. Node 2 sends a modified U-msg to all of its 
neighbors except the neighbor from which it received the 
U-msg. Now the U-msg is updated with the cumnmulative 
distance from node 2 to node 1. Let d(i,l) be the channel 
value on the link from a node ito any neighbor 1. fTfhen the 


Cummulative channel value from node 2 to node 1 is 
a(2,1) + D(I) = 1+028: 1. 


D (1) is taken from the U-msg received by node 2 from node 
a a(1,2) is calculated at some earlier designated time 
when all nodes in the network Simutanzously calculate and 
fix channel values to each of their neighbors (this 
procedure is discussed in greater detail in Chapter V). 
Therefore *+he U-msg relayed +0 node 2'S neighbors is 
U(2,1,1) which states that node 2 is relaying a U-msg frrorn 
node 1 and the cummulative channel value througn node 2 to 
node 1 along its best path is 1. 

When node 4 receives the U-msg from node 2, it once 
again processes the message in a standard fashion. As shown 
in Fig. 4.6, the channel value from node 4 to nodé 2 is 3 
eg) 3) =3) . Now upon receiving the U-msg from node 2, node 


4 caiculates the cummulative channel value through node 2 to 
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Pig. 4.6. Node 2 Relays Node i's U-masg 


mero nittiatcr of “the U-msg (d=nodel). Por node. Gan tals 


exanple 


Meceeeo +) (2) + 3 + 1 = &. 


W#hen node 4 compares this value with the latest cunmmulative 
channel value for its best path to node 1 (Which nod2 i will 
define as the symbol 8B (dj) Powe taueethat Lt iS nere 
Peeecien: <c go thru node 2 to get to node 1, or B(1)=2. 
Meee that iin future discussions the term "Best Path® will 
imply the optimum series of links, whereas 38B(d) will 
indicate a specific neighboring node which a transmitting 
node considers as the next downstream node on the best path 
moeacestination d. 
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In this example, node 4 receives a U-msg which 
enables it to improve its best path to d. Any node which 
changes its best path or the cummuiative channel value for 
its current best path must, in turn, relay this information 
+o all of its neighbors (except B(d). This is necessary 
because, given this new information, an upstream neighbor 
May hav? an opportunity to update its B(d). On the other 
hand, if a node receives a U-msg which does not change the 
node's B(d) or cummulative channel vlaue to d, it will not 
relay the U-msg. This is acceptable because the 'ipstrean 
nodes already have access to the current route which is 
considered more efficient than a route through the node 
which relayed the last U-amsg. 

Deletion of update messages 1s an SMpOrcant 
mmct ion. If the network were nct aliowed to eliminate 
useless nessages, it could impose a significant unnecessary 
burden on the management traffic load. In a network of N 
nodes, there are approximately N**2 best paths. Iz, when 
each node initiated an update operation, every other node 
indiscriminately relayed the update message, there would be 
@ minimum of approximately Nx(number of links) update 


messages generated when 2ach nod2 originates an update in a 
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network of N nodes. iierertole tO control this growth, the 
first node to receive a useless J-asg eliminates it. 

Fig. 4.7 shows the complete network with channel 
values and best paths from all nodes, to node 1, Ddefore and 
after update. The order in which U-msgs arrive at a node 


can Significantly affect the number of U-msgs generated ina 


reaching the optimum solution. But (assuming static values 
for the channel values) the end result will always be 
optimum, even though it may require several update 


initiation cycles to stabilize. The following is a sequence 
of events that could have occurred to update the network in 
meg. «64UT. 

Node 1 generates U(1,1,0) and sends it to Nodes 2 
and 4. 

Node 4 receives Node 1's U-amsg. Since this is 
Beeecady its 8B(1), 1% updates its net channel value, 
generates U0(4,1,5) and sends it to nodes 2,3, and 5. 

Meanwhile Node 2 receives Node 1's YJ-masg upstrean 
mmeomg 2¢Ss 38(1), updates its net channei value, generates 
U(2,1,1) and sends it to Nodes 3 and &. 

Node 2 receives Node 4's JU(4,1,5), compares it to 


Peery, and discards it. 
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Fig. 4.7. Complete network with Channel Values and BP(1)‘'s 
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Node 3 receives Node 4's U(4,1,5), compares it to 
imeeeilatest 8(1) and discards it. 

Node 5 receives J (4,1,5) Uestream 220m ats B(1), 
generates U(5,1,10) and sends it to Nodes 3 and 6. 

Now node 4 receives Node 2's U(2,1,1), compares it 
tO its last B(1) and selects a new B(1)=2. Since it changed 
B(d), Node 4 issues U(4,1,4) to Nodes 3 and 5 which will 
eventually be discarded by both nodes. 

Meanwhile Node 3 receives Node 2's U(2,1,1), updates 
its B(1) and generates U(3,1,3) for Nodes 4,5 and 6. 

Node 5 receives Node 3's U(3,1,3), finds this better 
than its previous B(1) and sets B(1)=3. Now Node 5 must 
also issue U(5,1,4) to Node 4 and 6. 

Node 3 receives Node 5's previous J(5,1,10) and 
discards it. Node 4 teceives Node 5's later U(5,1,4) and 
also discards it. 

Node 6 initially received Node 5's UJ(5,1,10) but 
meeained its old 8B(1)=3. Later Node 6 received J(5,1,4). 
Meeoecame it finds this path much better and sets 38(1)=5. 
Se also issues U(6,1,6)to node 3. Eventually Node 6 
receives 0(3,1,3) but discards it. Finally, Node 3 receives 


Moy t,o) and discards it. 
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In the above example, the senerio would have been 
Slightly changed had messages arrived ina different order, 
but the ultimate best path results would pe the same. 

2. Introducing New and Broken Links 

Realistically a network must integrate new links and 
recover from broken links. Later the "Alternate Link", as 
an interim fix, will be discussed. Bc I Ngeidalivewe shail 
assume that there are no known routes remaining from the 
node which detects the broken link, to some destination. 
Assume also that traffic for this destination nay already be 
stored in the detecting node, or enroute to it under the 
assumption that the broken sea Sono aan pact. The 
network management protoccl must provide for a graceful 
recovery. 

In order to eliminate the added complexity of cycle 
humbers, the pretocol is restricted to initiating one U-msg 
=<rtom any one node in the network at one time. This means 
that there must be enough time for an update cycle or 
session initiated bv a node to propagate thru the entire 
network, updating all best paths as it goes. When a break 
cuts off access to a node, it 1S important that a new update 


from that node (cr nodes) be initiated and propagated tairu 
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the network as soon aS possible in order to identify new 
best paths so that stalled traffic can continue to their 
destinations. In order to address this problen, the 
protocol assigns the highest processing priority to U-msgs. 
This is intended to allow U-msgs to perform the update (and 
therefore eliminate themselves from the net) aS soon as 
possible. At the same time, the protocol sets the frequency 
at which each node periodically initiates a new U-msg. The 
idea is to establish a practical U-msg initiation frequency 
so that the event of a broken link does not require a 
request for initiation of a special update message, and yet 
does not leave user traffic stranded for a long tine. 

It might be helpful to consider an example of this 
in terms of the user service protocol example in Chapter 
III. If the average distance petween nodes is approximately 
3 km (based on the Marine Amphibious Brigade acdel) then 
assuming speed of light propagation the signal travel time 
between nodes is 


i.e 
3 x 10%*%5 km/sec 





= 10%%*%-5 sec = 10 usec. 


Furthermore assume a network or basic group of 50 nodes, and 
assume a longest best path of 30 nodes. Then the maxinun 
total travel time is 
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30 links x 10 usec/link = 300 usec. 


Assume an additional 20 usec processing time in each node 
(when protocol messages are given top priority). Then the 
total time for a U-msg to process thru the entire net is 
approximately 1 msec. Therefore if the protocol required 
each node +o initiate a U-mSg every .1isec (or once every 10 
frames), approximately .1sec + 1msec is the longest any 
traffic sheculd be stranded due toa broken link. This 
Seould not significantly affect data traffic which is bursty 
in nature anyhow. Although detectable in voice traffic, it 
would not be serious unless failures occurred repeatedly. 
This situation could be improved by increasing the frequency 


of the update at the cost of more network management 


Mrs | PpRetecc. Gequires that teartlic . Which is 
Stranded due to a broken link wait to be rescued by a 
routine U-msg from the destination node to which the traffic 
is addressed. Yet there are still actions which can be 
taken to make gocd use of the proken link information and 
minimize the rauma of recovery. Since there .is. no 
assurance that any cf the nodes close to the break will be 


On the new best path, it is probably helpful to freeze data 
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Mmenee1c 6eNrOute £60 4 a broken link. This is one of the 


functions of a broken path message (X-msqg). 


Poe eatin link to node 1 
Be 





Pig. 4.8. Sending the X-msg Upstrean 


When a node discovers a broken dlink on cne cf its 
best paths, it initiates a X-msg for every destination node 
for which the discovering nodes considered the broken link 
as part of the best path. These nodes are easy to identify 
because ~his is the same information used for normal routing 
Operations. fwe itiatiatang mode sends the X-msgs to all of 
its neighbors. For example, as shown in Fig. 4.8, when Node 
3 discovers that the link to Node 2 (and B({1)) is broken, it 


fee anitiate an X~msg which is X(1). Note in this example 
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+hat the broken link could also be Node 3's 3(2), also 


requiring a X(2) message. But for simplicity it is assumed 
that Node 1 is the only destination in this network. The 
X¥(1) is sent to all of Node 3's neighbors. Node 4 and 6 do 


not consider Node 3 to be on their best path to Node 1. 
Note that for Node 4, this is a correct assumption. But for 
Node 6 this assumption is not correct. In any event, if a 
node receives a X-msg froma non-best path neighbor, it 
discards the X-msg and takes no other action. This is how 
useless X-msgs are eliminated. 

Nome, 5S, son tne ocher hand, ceceives X{1) £zom its 
B(1). This indicates that it has lost its best path to Node 
faeaAS With Node os when a node discovers that its best path 
memeeis broken, it freezes any data traffic in its buffer 
FOr d, and issues an X-msg. Therefore Node 5 now issues 
X(1) to Nodes 4 and 6. Once again Node 4 ignores the X-msqg. 
However this tine Node 6 has received the X-msg from its 
beat). This wouid cause Node 6 to stop sending data traffic 
Mmeert a new best path is found. 

At the User Protocol level, which may employ virtual 
links as described in Chapter III, the X-msg may not be 


seman =O stop all traffic routed over a given link when a 
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break is discovered. beaver ink faxes a route at the 
meme it iS Created, for the duration of the traffic session. 
In the meantine, subsequent update cycles may have caused 
nodes along an established virtual link to select other 
nodes as B(d) while maintalning its virtual link thru the 
node which was the B(d) at the time the virtual tiink was 
created. Thererore, in addition to sending X-msgs to all 
neighbors for all destinations for which a broken link was 
considered a best path, 1t may also be necessary to define a 
Virtual Disconnect message which would be relayed upstrean 
to break down virtual links. In fact something lixe this 
would probabiy be reguired in any network uSing virtual 
links to break down the virtual links when users have 
completed a routine traffic session. 

Because of the frequency of the U-msg, it may not be 
necessary for a X-msg to work its way all the way upstrean 
to the most remote node on the best path. In Fig. 4.9 Node 
2 could have issued its X(1) after Node 1 issued U(1,1,0) to 
Node 4, ieee s Case, if Node 4 had not yet processed 
U(1,1,0) when it received the X(1) from its B(1), Node 4& 
would immediately adopt the direct link as its B({1) and 


issue 0(4,1,5) te Nodes 2, 3, and 5. Since Node 2 already 
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Fig. 4.9. X-msg Meets U-asg 
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meeee traffic for Node 1, 1t would immediately release the 
traffic to Node 4 considering Node 4 as its B(1). If Node 3 
Mmeaeaiready frozen traffic for Node 1, the same would apply. 
Bux in this situation it is linkely that anew best path 
would be established before traffic in Nodes 5 and 6 were 
even affected by the braken link. 

New links do not have the traumatic impact of broken 
links. A new link represents the addition of a new neighbor 
meee ch= two nodes on each side of the link. Since the link 
is initially unloaded, it is likely to become a prime 
Candidate for a link in several best paths because of its 


low channel value. It is necessary to guard against 
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oscillations here which can be done by asSigning arbitrary 
initial average channel value to the new link which would 
enable the link +o be gracefully integrated into tne 
network. In time, as the link becomes used, the effect or 
this arbitrary asSignment will disappear. Once again, 
because of the frequency of initiating U-msgs, there is no 
need to request special updates upon the discovery of a new 
link. It will be integrated into the network guickly enough 
just by normal updates. 


Bee Alternate Link = an Interim Fix 





It is not always necessary to stop traffic in the 
face of a broken link. Ideally every best path would have a 
backup path so that when a broken link is discovered, 
traific is immediately Switched to the backup oath with 
minimum cipple in network trafrfic flow. But this may not be 
possible, and the additional complexity in the protocol as 
Well as the increase in the volume and content of network 
Management messages appears significant. 

However, as explained in Appendix A, the protocol as 
described in this chapter provides sufficient information to 
manage the basic update and broken link functions, and with 


a Slight increase in processing at each node, the same 
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network management messages can occasionally provide a 
real-time routing alternative to a broken link. Deas 1S 
called an alternate link. 

The alternate link is identified during the update 
operation. For example, during a routine Update for Node 1 
of the network in Fig. GSO, Node 2Zewill send U(2,1,2) to 
Nodes 3 and 4. Node 4 will not select Node 2 as its B(1), 
and would normally discard the U-msg. However if Node 4 
made on2 additional comparison, it may still find the Node 2 


route to Node 1 useful. 





Pog. 4oude tne Alternate Link 


For any node j, by comparing the cummulative channel 
value (D(1)) of the last relaying node (1) with node j's 


current cummulative channel value along its best path to d, 
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node j can determine if traffic passing thru node 1 can also 
eventually pass back thru node j. Assuming all links have a 
minimum channei value >0, if node j's cummulative best path 
Channel value >= D(1), then node j is assured that node l 
does not pass traffic thru node j in order to get to d. 
This minor conclusion provides node j with a loop-free 
alternative path to d if it should discover a preak in its 
best path. Thas alternative says nothing about 
multi-destination or implied loops. ELeNOniky “OELersS a4 
temporary fix for anode which has experienced a broken 
Peon kK. 

Going back to the example in Fig. 4.10, Node 4 notes 
an U(2,1,2) that D(2) = 2 which also equals the cummulative 
Ghannel value for Node 4's B(1). This causes Node 4 to list 
Meage 2 as an alternate link to Node 1. In larger networks, 
one node can certainly have alternate links for several d's 
as well as several alternate links for a single d. Node 3 
in the example sets B(1) = Node 2. However when it receives 


Wee, 2), it Will list Node 4 as its aiternate link to Node 


ie Likewise Node 2 will pick Node 4 as its alternate Link 
20 Node 1. But note that Node 4 can not rely on Node 3 as 
ema cernate link. When Node 4 received U(3,1,4) from Node 
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3, it has no way of insuring that the route is not as shown 
me Fig. eels Therefore inthe absence of any further 
evyerhead traffic, Node 4 discards this information as 


unreliable. 





Fig. 4.11. Potential Loop Situation 


The impact of alternate links is not clear. Le 4 
network is very evenly weighted and richiy connected, #2ach 
node could have one or more alternate links to most of the 
other nodes in the network. This implies that broken links 
May only require a shift in traffic. A less eveniy 
distributed network would have some nodes with alternate 
links and others without. In this case some X-msgs might be 
avoided, others curtailed, and yet others unaffected. At a 


Minimun, the alternate link concept appears to add 
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additional robustness to the network. At best it may ailow 
+he Update frequency to be decreased, cutting down the rate 


of management traffic. 


De EXPANDING TO RELATED GROUPS AND FAMILIES 

Although there is no theoretical limit on the number of 
nodes oe | basic “Group, there may be practicai 
considerations which make it attractive to limit this 
number. For example, when all nodes are considered part of 
a Single basic group, then every node in the basic group 
(which includes the entire network) must record a B(d) for 
every other node in the network. This further implies that 
updates for every individual node can potentially span the 
entire network. As the number (N) Gf nodes 2m) Ya saenly 
connected network grows, the number of U-msgs generated (to 
Semep. ete an update for one source) under worst-case 
conditions approaches 3 (N*#2). (See Appendix ¢C). Thererore 
it may be convenient to partition the network along 
operational or geographical boundaries. To investigate 
Ses, several additional definitions must be intrcduced. 

1. More Definitions 

Mmeemce co D alt OL Fig, “S712, all the nodes in a 


given network fail into one of six groups numbered 100 thru 
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600. Nodes within a particular group consider that group 
mes basic group. Within a baSic group each node has a 
unigue node identity. For example, in Group 400, there is 
only one Node 2. Outside of a node's basic group are other 
groups. For each group in the top of Fig. 4.12, there are 
five other groups called Related Groups. Note that related 
does not imply that two groups have a common border. FOr 
example, Group 400 does not border Group 200. By combizing 
the group and node ldentity, each node can, once again, have 
a unique identity in the network. For example, Node 2 ia 
group 400 can uniguely be called Node 402. 

muGthesmwoLre, the six groups in the top of Fig. 4.12 
San be grouped together and called a Family. This family 
could also have a unique identity, such as 3000 in Fig. 
4.12, and be one of a number of families which combine to 
momm 2 large network of nodes. In the bottom of Pig. 4.12, 
there are four families numbered 1000 through 4000. Once 
again, every group, in every family in the bottom of Fig. 
Mme « COUld have a Node 2. But when group and family 
identities are added to the node identity, each node retains 
a uwunigue identity. To fully identify Node 2 mentioned 


earlier, it can now be called Node 3402. By uSing this 
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three tier identity concept, there 1s a potential to reduce 
network management traffic. There is no requirement to stop 
at three tiers; however three tiers suffice to demonstrate 
the principles. 

One requirement of this structuring principle is 
that groups and families must retain some continuity. That 
does not mean that groups and families cannot move in 
relation to each other. It simply means that entire groups 
and families can not distribute all their nodes randomly 
around the network. If the network must tolerate complete 
crandcm node movement, then the single basic group concept 
seems best suited to control the network. However there may 
be Several situations wherein clusters cf nodes are likely 
to remain geographically and operationally close while being 
fluid ina larger network of nodes. Military organizations 


are a good example of this structuring. 


y 


ae £ficiencies of Grouping 


C 


In the remainder of this study, any reference to 
node identity will imply the full identity including the 
node's basic group and family, if applicable. All message 
formats and contents are also the same. It will be assumed 


that all nodes know their assigned group and family. In a 


78 





military network for example, the group could represent the 
battalion, and the family could represent the Brigade. 

As shown in Fig. 4.12, individual nodes continue to 
establish links With neighboring nodes regardless of 
arbitrary boundaries. Efficiency is available by changing 
the processing of messages that cross these boundaries. The 
goal is to reduce the number of network nanagement messages 
that travel to remote nodes in the network when there ls 
small likelihood that the best paths being updated by these 
messages will ever be used. 

Basic groups are organized *to contain a group of 
nodes which communicate frequently with each other. Every 
node in the basic group has a best path to every other node 
maecae basic group. For these nodes, which constitute a 
mini-network, the basic network managemenz bEGteCcOL 
described in Section C above applies directly. However, the 
node to nodé protocol will establish a link with any node it 
eemecontact. Therefore a node may find that it has a link 
with another nede outside of its basic group. This is where 
group/family processing begins. 

Fundamentally, grouping causes nodes to treat 


celated groups and related families as single nodes, while 
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still maintaining the capability to contact each node in che 
network. Therefore for our exampie in Fig. 4.12, Node 3402 
has two other nodes in its basic group, five related groups, 
and three related families. Thus Node 3402 maintains a best 
path toa total of 10 network elements. By contrast, 
without groups Node 3402 would be required to maintain best 
paths to 57 individual nodes in order to contact every node 
in the network. It should be noted that basic groups of 
only three nodes is probably unrealistic. Basic groups of 
feeto 25 nodes, families of 3 to 5 groups and networks of 3 
£0 5 families would fit typical military organizations. 
Although there is probably an optimum combination for a 
meen traffic profile, there are no rigid requirements on 
grouping sizes. 

During the course of a normal Update, an initiating 
node, or a relaying node will send a U-msg to all of its 
neighbors. The receiving node ({j) checks the identity (1) 
of the node which last relayed tne U(1,d,D(1)) message. ie 
l is not in Node j's basic group, and if 1 does not equal d 
Meoecating the last relaying node did not initiate the 
message), Node j discards the U-msg. If l=d, this indicates 


*o Node j that a neighbor outside Node j's basic group has 
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initiated an Update. If the neighbor is from a related 
group (same family), Node j compares i*s cummulative best 
path channel value for that related group to the net channel 
value through l. If this is an improvement, Node j alters 
the U-msgq with its d(i,1l), and relays the U-msg to all 
neighbors. This procedure continues until this U-msg can 


not offer an improved best path to any other node or until 


it reaches the family boundary. If Node j's previous best 
path was superior to the new possibility, Node j would 
discard the U-msgq. If the neighboring node (1) which 


initiated the message was from another famiiy, node j would 
check its current cummulative best path channel value to i's 


family, and compare it to the net channel value through l. 


AS in the case of a related group, 1f there is an 
improvement, the U-msq is relayed, otherwise it is 
discarded. 

Fee ie 4.13. serves to illustrate Updates across 
boundaries. The process may become clearer by tracking a 


possible sequence of events during a routine update 
Operation. In Fig. 4.13, the dotted triangles pointing away 
from a node represent that node's (pre-update) best path to 


a tvelated family (if the U-msg creating the best path had 
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Update initiated by Node 1301 


| 





Ie 


m= PREVEOQUS Best Path WB BP after Update 


Fig. 4.13. Update Across Boundaries (partial network) 


crossed a family boundary after being initiated) or best 
path to a related group (if the U-msg creating the best pata 
had crossed a related group boundary after being initiated). 
The dark triangle represents the updated best paths after 
Node 1301 issues an update. 

When Node 1301 issues a U-msg to all of its 
neighbors, the nedes in basic group 1300 will update like 


eye Dasic group. Nodes 14203 and 2202 will also receive 
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U (1301,1301,0). Node 1203 sees that this U-msg was 
initiated by one of its related group neighbors, and after 
comparing channel values with its old best path through Node 
1202, selects Node 1301 as its new best path to Group 1300 
(or B(1300) =Node 1301). This also requires Node 1203 to 
adjust and relay the U-msg to all of its neighbors. Node 
2102 receives Node 1203's U-msg across a family boundary, 
notes that it was relayed but not initiated by Node 1203, 
and discards it. It is discarded because this particulac 
version of Node 1301's four original update messages (one 
for each link) initially crossed a Group boundary. 
Therefore all subsequent versions of this U-msg serve to 
update best paths Cemmarcoup 1300@Gwithin Lts family. 
Therefore when Node 1203 relayed an offspring of the "group" 
version outside its family, Node 2102 discarded it. Node 
1202 keeps its best path to Group 1300 through Node 1303. 
Node 1201 changes its best path through Node 1203 and relays 
the U-msg to its neighbors. 

Now Node 1103 notes that a Group 1200 node has 
relayed an U-msg initiated by a third related group in Node 
1103's family; therefore it will evaluate this message in an 


2ffort to improve its path to Group 1300. Note that once a 
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U-msg successfully crosses a group boundary leaving its 
basic group, it continues to cross other group boundaries 
until it no longer cffers a shorter best path, and is 
discarded. 

The same considerations apply when lnitially 
crossing a family boundary, as will be seen below. Node 
1103 updates its best path to Group 1300 and relays the 
U-msg to Nodes 1101 and 3102. Node 1101 retains its path to 
Group 1300 through Node 1103. It so happens that Node 3102 
currently has Node 1103 as its best path to the 1000 Family. 
However when it receives the U-msg relayed by Node 1103, it 
notes that it was not initiated by Node 1103 and discards it 
Since Node 3102 is net interested in establishing @ path to 
Group 1300, or any other individual group inthe 1000 
Family. 

Meanwhile Node 2202 has also received the U-msg 
initiated by Node 1301. Node 2202 updates its net channel 
value retaining this best path, and relays 
Meeez02,1301,D(2202)) to ali of its neighbors. Node 2102 
accepts this new route as its best path to the 1000 Family 
in preference to its less efficient link through Node 1203. 
It then relays the U-msg to its neighbors. Node 2101 


updates and relays again. 
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Now Node 3102 has again received a version of the 
U-msg initiated by Node 130%. But this time it was passed 
by a node which was not in the 1000 Family, indicating that 
the cummulative channel value in the U-msg up to this point 
represents the distance along this proposed path to the edge 
of the 1000 Family. Node 3102 er this to its current 
best path channel value to the 1000 Family (which is direct 
#o Node 1103), and picks the best path. In this example, 
Node 3102 found that it was more efficient to travel to the 
1000 Family through the 2000 Family, than to cross the 
direct link to Node 1103 (rather unusual). 

To use this routing information, the Source node 
addresses traffic to the destination node and sends the 
traffic on its way. If the destination is in the same basic 


group as the source, the source has a best path direct to 


the destination nede. If the destination is ina related 
group (same Family), the source node sends the traffic on 
the best path tc the destination node's basic group. As 
soon aS it crosses the basic group boundary, oer cleat. vc 


will reach a node which now has a best path to the specific 
Mesceration node. Similarly for inter-family traffic: it is 


routed on the source node's best path to the family of the 
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destination. When it cresses the family boundary, 2692S 
routed by the destination's group and finally when it 
crosses the basic group boundary, it is routed to the 
specific node. 

The cost in routing inefficiency entailed by the 
tremendous reduction in overhead traffic offered by this 
scheme is cbvious from the example in Fig. 4.13. If Node 
3102 wanted to communicate with Node 1101 after the Node 
1301 update (dark triangles), the traffic would ultimately 
travel through nearly every node in the figure, when iz fact 
Node 1101 is only two links away from Node 3102. Although 
it has been established that it is shorter to go from Node 
mee to Node 1301 than to Node 1103 in this case, the full 
trip would normally be shorter by the more direct route. 

Besides the reduction in overhead traffic, it should 
also be noted that group and family boundaries would 
normally be selected on operational boundaries, so thata 
relatively small amount of traffic would be expected to 
Suffer from this self-inflicted inefficiency. 

There are some minor exceptions to the above rules 
which would be helpful if integrated into this scheme. For 


example, any node on a boundary with a non-best pata direct 
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[meme tO a node in ancther group or family, need not reject 
Or purposely break this link simply because it is programmed 
only to use that node's group or family address. The minor 
additional overhead of retaining Genees Laks “<o all 
neighboring nodes can be useful in recovering from broken 
links. Both the broken path and alternate link concepts 
described for the basic group can be applied, virtually 


unchanged, to the group/family processing concept. 
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Figure 4.14%. Broken Paths and Alternate Links Across Boun- 
daries 


A broken link on a best path to a related group or 


memetly causes the discovering node to first look for an 





alternate link. ear Gg). 4.14, a portion of a network 
including boundary crossings and link channel values is 
shown. If these channel values had existed when Node 1203 
last updated, Node 2102 would have retained its B(1000)=Node 
1203. However when Node 2202 relayed YJ (2202,1301,3) to Node 
2102, Node 2102 would see that D(2202)=3 to the 1000 Family, 
which is less than its cummulative best path channel value 
to the 1000 Family. Therefore Node 2102 would keep Node 
2202 as an alternate link to the 1000 Family. Then if Node 
mee Jost its direct link to the 1000 Family, it could 
immediately switch traffic to Node 2202 with the assurance 
meas traffic would not enter a loop. Conversely, Node 2202 
mod NOt pick up Node 2102 aS its alternate link to the 
1000 Family because Node 2102's cummulative channel value 
(4) ius greater than Node 2202's cummulative channel value 
fa) . 

Tf Node 2202 experienced a broken link to the 1000 
memely in Fig. 4.14, it would be required to initiate a 
Droken path message to all of its neighbors. When Node 2102 
received X (1000) frem Node 2202, 1+ would disregard tne 
X-msg since its best path is not affected. When Node 2204 


received x(1000), it would find that its B(1000)= Node 2202 
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indicating it had lost its best path to the 1000 Family, and 
Yook for an alternate link. If during the last update by 


Node 1203 the channel value b in Fig. 4.14 were such that 


b + Node 2202's D(1000) >= Node 2102's D(1000) 


or 
b+ 3 >= 4, 


*hen Node 2204 would switch traffic for the 1000 family thru 


Node 2102. If 
b+ 3< 4, 


then Node 2204 would relay the X-msg or X(1000) £O all oft 
its neighbors. And the process would continue outward fron 
the broken link just as within a basic group. 

In this discussion, it should bé noted that the 
baSic group concept of network management protocol can be 
applied directly to the group/famiiy organization of the 
network, with scme reguirements on the structure of the 
group and families. The idéa of detached nodes in the 
Meeup/famiiy concept is a speciai case which wiil be 
discussed in Chapter JVI. Whether or not tne group/family 


concept should be imposed on the network is a function of 
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network size, ¢he User traffic profile and efficiency 


m@cadeoffs. 
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Ve. SIMULATION 


A primary objective of the simulation was to test the 
basic algorithm by selecting and fixing some network 
parameters, and then making multiple runs in which the 
remaining parameters were varied. Though limited in scope, 
the simulations validated some of the mechanics of the 
algorithm. This included originating and relaying update 
messages, which further resulted in selecting and updating 
best paths based on calculated channel values. Both the 
basic group and family/group concepts were tested. Two 
methods of calculating channel values, both using a variable 
time duration called a window, were also investigated. The 
test network is shown in Fig. 5.1. 

Simulation results were initially compared to results 
Obtained using static routing via fewest number of hops over 
the same network. Later, selected parameters were varied to 
Observe the stability and robustness of the network control. 
AS a result, several basic observations were made about the 
attributes, efficiencies and limitations of this management 
eeococol concept. . 

The broken link and alternate link concepts were not 


Beee Of this initial simulation. EE che Vbasic Link 
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family boundary 


SS ae group boundary 


is 


Fig. 5.1. Test Network 


management concept ultimately proves to be worthwhile, as 
these initial tests suggest, tren cae Next logical step 
would be +o test the algorithm under the added strain of 


gaining and losing links. 





The simulation was conducted using the SIMSCRIPT II.5 
Simulation language. The encoded algorithm is listed in 
Appendix E. Several SIMSCRIPT encoding decisions are 


discussed later in this chapter. 


A. SIMULATING A USER AND MEASURING EFFECTIVENESS 

In order to observe and measure the relative 
effectiveness of the algorithm, a simple user service 
protocol involving only data packets was integrated into the 
systen. User traffic sessions were generated with an 
exponentially rcandom inter-arrival rate and with a uniformly 
random number of packets. Both the rate and number of 
packets were controlled by input variables. A packet either 
moved thru the network, or walted in a queue if the required 
[meee was busy, until it arrived at its destination where it 
was discarded after performance data was collected. All 
traffic sessions (and therefore all packets) had 4 source 
node determined by a uniform random function based on a 
transmit factor assigned to each nede. Each packet also had 
a destination assigned by a similar process. Packets 
created in a single traffic session all had the same source 


and destination. 
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One measure of relative efficiency was the average time 
(per total nodes hopped) hee tOGkeapackees tO reach their 
destination. Other, perhaps more significant, measures of 
effectiveness involve the- amount of queuing delay or guéue 
sizes that occurred during the test. This was observed in 
several ways. 

The maximum gueue size per Simulation was trecorded for 
every link and listed after every run. This information 
varied significantly and appeared to be influenced by the 
Meege  .ntlux of packets during the initiation of traffic 
sessions. 

Half way thru the simulation, a group of links having 
the longest queues during the first half of the test were 
selected +o be sampled during tne second half of the test. 
The number of links selected was an input variable. The 
number of samples per links was also an input variable, but 
Was normally set at 1000. The resulting distribution of 
Sample sizes for the busiest links in tne network, appeared 
momecrter a stable, more representative measure of the 
algorithm's ability te process packets. The average sample 


queue size and its standard deviation was also calculated. 
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Other checks and measures included the averag? number 
of nodes hopped per packet, the average number of links used 
for a node's update cycle, and the longest best path 
established anytime during the test. Finally there are 
several checks to report it packets were excessively 
delayed, particularly due to dynamical changes in best paths 
during message transmission, resulting in an abnormal number 


of hops to the destination. 


Be. PROGRAMMING SCHEME 

The simulation program was organized as a sét of 
Peosoetttines controlled by a Simulation clock (Fig. Dae) 
Which is an inherent feature of SIMSCRIPT. Before the 
Simulation begins, the routine is initialized by the main 
program which includes reading input variables, dimensioning 
arrays and printing out various input parameters. The main 
progran also schedules the events on the simulation clock 
which starts several activity chains re¢sulting in the 
generation of user ‘traffic, the periodic update of the 
network and the collection of performance data. 

The Main” program schedules the first update originated 
Dy each node in the network. This is begun at a random tine 


thru an event routine called NEW.UPDATE.MESSAGE. For the 
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Pig. 5.2. Program Organization 


designated node, this routine generates a U-msg for each of 
its neighbors and places the messages oon the link to each 


Memghbor. The routine scheduies the arrival of each U-amsg 
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after a pause to account for propagation time plus message 
duration. A time of 2ms was selected for the simulation. 
Finally this routine reschedules the designated node for its 
next update origination in an interval which was based on an 
input variable explained below. 

Once a U-msg waS originated, it was scheduled to arrive 
at neighboring nedes. The arrival of a U-msg was handied by 
a routine called ARRIVAL.MESSAGE. This routine 1S the heart 
of the update operation and implements the update portion of 
eee algorithm in Chapter IV. The ARRIVAL.MESSAGE routine 


determines whether cr not this U-msg should be rélayed to 


*he neighbors of the receiving node, which neighbors to 
Gemay it to, and what the contents of the relayed message 
will be. It simulates the processing time by scheduling 


retransmitted U-msgs to continue after a brief processing 
*lime. U-msg processing tine was set at 0.1 x« the packet 
processing tine (.0001 sec) to reflect the priority of 
U-msgs and their small relative size. 

After the processing time, the U-msg is placed on the 
next link by the CONT. UPDATE.MESSAGE routine and the U-asg 
is again scheduled to arrive at the next node in the 


selected transmissicn time of 2ms. This process contiaues 
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until the U-msg arrives ata node, is processed by tae 
ARRIVAL.MESSAGE routine and considered no longer Suitable 
for retransmission (due to an excessive net channel value). 
The result is the creation of a best path to the node (group 
or family) which originated the U-msg from every other acde 
thru which the message successfully passed prior to discard. 

During the course of the simulation, as link queues vary 
in size, channel values change. One of the most Significant 
observations affecting the fundamental algorithm made during 
the Simulations, concerns the timing of when channel values 
may be calculated. It was initially conceived that during 
an update cycle, a node could calculate its channel vaiue to 
a neighbor whenever that node received a U-msg from that 
neighbor. However it was found that under a relatively high 
traffic rate, scme node (i) Might relay U-msqs aaving 
selected a best path node (3), but by the time node i 
received relayed versions of its own U-msg its channel value 
to node j might have changed dramatically, resulting ina 
Meop (See Fig. 5.3). To remedy this probiem, updates were 
Semsterained oO start anytime during a (relatively large) 
time interval. This interval was followed by another equal 


Size interval during which no updates could be started, but 
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existing updates could be processed. The minimum size of 
these intervals was large enough to insure that any existing 
update cycles would work their way out of che network before 
+he next series of updates was allowed to begin. The 
Calculation of channel values was synchronized in each necde 
to take place once (and only once) near the beginning of the 
first (originaticn) interval. The operational feasibilicy 
of this synchronization requirement iS not unreasonable, for 
very good network synchronization will be a likely 
requirement in crder to take advantage of the benerits of 
Spread spectrum modulation technigues, position location or 
other attractive capabilities of digitai communication. 

In the simulation, the Main program schedules the first 
channel value calculation with the routine CV.LATCH. Since 
*his takes place at time zero of the Simulation and no 
traffic has started, all links are initialized to the basic 
Ghannel value of 1. CV.LATCH also calculates the update 
Origination interval, based on the specified update interval 
which is an input variable (O0P.DATE.PERIOD), and reschedules 
ltself for every node in the network. 


Ae 


(D 


r the first update, the CV.LATCH routine uses 


historical queue information for each link to calculate a 
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Fig. 5.3. Possible Result of Frequent CV Changes 


mew Channel value for that link. This value is based on a 


time average of past gueue sizes existing at that node over 


2 time period called the WINDOW. The channel value is the 


be 


mreeger part of 


ap ely 
WLNDOW 


Q; = queue size of i th gueue 
4 mee 


ime interval over which queue = Q; 
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summed for all queue measurements not older than the WINDOW 
mor a given link. 

The Wein progam begins to scheduls traffic with an 
2xponential arrival rate based on an input variable 
eee o NEW. TRAFFIC .INTERVAL}). Traffic is started by calling a 
routine called NEW.PACKET. MESSAGE. mae f2nSewtresiic is 
generated after the network is allowed to complete one 
update cycle, thus insuring that all nodes have a best path 
to the other nodes, groups or families as appropriate. A 
traffic message (referred to as "session" in Appendix &) 
involves randomly selecting a source node, destination node 
and the number of packets in the agessage. The selection of 
*he nodes is a function of input variables assigned to each 
node which dictate the relative frequency with which nodes 
fee cloansmit and receive. Rie SOuUcL ne Can aiso restrict 
destination nodes to be in the same group or family as the 
source node for a given percentage of the traffic messages 
(sessions) based on additicnal input data. The routine will 
send the first packet on the best path to its destination if 
M@ewtink 1s idle. If aot it will place the packet in a link 
queue. In either case, all other packets in the message are 


placed in the queue. 
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When a packet leaves its source node, it is assigned the 
current simulation time which 1s checked again upon arrival 
at the packet's destination. This information is used to 
comput? the average and peak times for packets to hop N 
nodes. Each packet counts the hops or nodes it passes thru 
enroute to its destination as explained later in this 
section. 

When a packet leaves for its first best path neighbor, 
it is scheduled tc arrive after an interval representing the 
packet transmissicn time, which is an input variable called 
eee MN.TIME. ror the simulation this value was fixed at 
50ms, based on performance factors mentioned in Chapter III. 

Finally NEW.DACKET.MESSAGE reschedules itself for the 
next traffic session which will have the same exponential 
inter-arrival rate mentioned above, but will result in the 
Candom selection of anew source, destination and message 
size (number of packets). 

Enroute +0 ences Tdestanatlon, packets arrive at 
neighboring nodes which is Simulated in the ARIVE.PACKET 
moewrts ne, Potrero cOUtT=ne a Gacket 25 checked to see if it 
has reached its destination. He Someit is processed ina 


routine called COMPLETED. TRIP discussed below. Revnot the 
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packet is processed; routed to the next best path neighbor 
based on the ID of the family, group or node of the 
destination node; and then either forwarded if the link is 
idle, or placed in the link's queue. ARIVE.PACKET schedules 
2ach arriving packet thru the CON.PACKET routine after a 
processing tine delay which was a4 test parameter fixed at 
0.1mSec per packet. Finaily ARIVE.PACKET goes back to the 
queue of the node which sent the last packet. If another 
packet is in the queue, it is placed on the link (by 
scheduling an ARIVE.PACKET for that packet) to the node 
which jus*+ received the last packet. If the queue was 
empty, it is designated as idle. 

Meanwhile, when the packet scheduled for the CON. PACKET 
meer ne arcives, if the link to its next node is idle, it is 
placed on the link and scheduled to arrive (ARIVE.PACKET) at 
tne next node in the packet transmission time (PKT.XMN. TIME) 
mentioned above. If not, it is placed in the queue for that 
teen K . In order to minimize large-scale loading shifts from 
one link to another, the algorithm does not change the 
routing of a packet coming out of a queue to be transmitted 
if, during the time the packet was waiting in the queue, the 


best path to its destination has changed. A packet keeps 
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its criginal routing unless the link has been broken (not. 
covered in these simulations) and newly arriving packets are 
routed thru the best path node. 

Eventually the packet reaches its destination. Here it 
is processed by the COMPLETED.TRIP routine. This routine 
collects and computes performance data including the number 
of nodes hopped by the packet, and trip time. * increments 
a counter which sums the number of packets hopping N nodes 
and records t«he highest trip time for N nodes. It keeps 

rack of the number of packets arriving for each session and 
Sums all the trip times for N nodes so it can later be 
divided by the total number of nodes making N hops to 
calculate the average trip time for N hops. 

After four equal intervals (quarters), the simulation is 
Stopped with the STOP.SIMULATION routine. This routine 
reprints selected input data. It also calculates and/or 
prints performance data for the simulation up to that point. 
Appendix E contains an example of the full ovrintout which 
includes *+he average and peak packet transit tine for N 
hops, and the maximum queue for every link. It also 
presents results of a statistical sampling of the links with 
the largest maximum queues during the first half of the 


Simulation. 


104 





The Main program schedules the QU.SAMPLER routine at the 
mid-point of the simulation. This routine will identify the 
M links with the highest gqueues in the first half of the 
Simulation. M is an input variable (SMP.LINKS). QU.SAMPLER 
then schedules a routine called SAMPLE which samples these &M 
links in the second half of the simulation with an 
exponentially distributed time between samples with aean 
1/S, where S is another input variable (NO.OF.SAMPLES). The 
queue sizes found during these samples increment @ queue 
size counting array called QU.DISTR. STCP.SIMULATION prints 
the results of this queue sample (QU.DISTR) as weil as 
calculates the average gueue size and its standard 
memiation. After fcur reports STOP.SIMULATION halts che 


test. 


Bee aRAYS AND TEMPORARY ENTITIES 

SLaSCRIPT is an excellent programming language, 
particularly for its readability and simulation oriented 
marmot lons. The encoded algorithm and related routines in 
Appendix E are written in SIMSCRIPT and also have additional 
documentation. However the organization of the arrays and 
attributes of the "message" and "pack" temporary entities 


contain several subjective encoding decisioas. 


105 





Understanding these organizational decisions will heip when 
reading Appendix E. 

The arrays used in the program are listed in Fig. 5.4 
There are one, two and three dimensional arrays 
(1=D,2~-D,3-D). The array name is followed Dyeeits 
dimension(s), which may either be variable or constant, and 
by the different meanings for the subscripted variable (é.g. 
node ID). Below each array name is the meaning of the first 
(1-D), second (2-D) or third (3-D) subscript. Together the 
subscripts identify a variable location which nay be used 


during the Simulation. 


EOL example, the FL ES array (FAM.OfF.GRP) 1s 
1l-dimensional. Its size is the sum of the number of nodes, 
plus the number of groups pius 25. Arguments of this array 


will be «he program numbers ‘for groups (program numbers are 
2xplained in Appendix £). The content of this subscripted 
Variable is the family of the group in the arcgument. 

The 2 and 3 dimensional arrays are read similarly. For 
seeample SMP.SET is a 2-dimenSsional array. Ehe see rst 
argument is the count number of the iink to be sampled which 
is determined by the QU.SAMPLER routine. The second 


argument identifies whether the variable is the "to" or 
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=) 


FAM.OF.GRP (no. of nodest grpst+ 25) -~> family ID 
Ist-D (program ID for group i) 


QU.DISTR (250) --> sample count 
Ist-D (queue size) 


Z2=D 


LINK.ABLE (no. of links in network, 2) --> node ID 


Ist-D (link number) . 
2nd-D 1= 1st nede j 2= znd node) 


TRACER (2 x test duration/ave. session intervai, 2) 
--> no. Of pkts 
Ist-D (session ean 
2nd~D 1= original pkt count for this session 
| 2= pkts which reached destination) 


CLOCK.DATA (4 x no. of nodes, 2) --> time 
1st-D (no. of hops = _N) 
Zna— 0 1= oy oe for all pkts hopping N nodes 
2= pe oe Seer eo crap time Lor 


hop pk 


HOP.COUNT (4x no. of nodes, : = Oe eae S 
1st~-D OE eee he ee = N) 
Papal Koad 8 sani 
3= no. of pkts hopping N hops) 


meen (ad selected no of Hinks, 2) -~-> node ID 
1st-D (sample link ID ioe | oe 
2nd-D 1= "from" node ID = "to" node ID) 


Fig. 5.4. SIMSCRIPT Arrays (1 and 2 Dimensional) 


ioeeom’ node for that link. The subscripted variabie is the 
@ecual identity of the node. 

Pinally for the NEIGHBOR.LIST array, the first argument 
is a Simple counting integer corresponding to one of the 
above node's neighbors (a node may have up to 6 neighbors). 
The third argument describes whether the subscriptred 
Variable will contain the ID of the neighbor node (1), an 
integer (1 or 0) indicating whether or not the neighbor is 
active (2), or tne channel value to this neighbor (3). 
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a2 0 


MSELGHSOR. LIST (momet nodes, oh 
eae node ID or "tp CWN or CY 
ist-D Pacn n's ID) 
2nd-D (a4 number listing from 1 to 6 of 
cf node n's neighbors} 
3rd-D { i= neighbor ID 2= link status. 
{| 3= CV £Erom node n to neighbor) 


BEST.PATH (no. of nodes, no. of nodes+ Se 
families, 2 --> node ID or 


Ist-D (“from"™ node x ae 
ae Ntow ip Cl go ther BOSC group or famiusy) 
1G (Dike 


1= reo path neighbo 
| 2= Gy thru best Pac ceth neighbor) 


LINK.MONITOR gee or BeGes « no. Or nodes, 3) 
--> rae ignai (1) or Q Size 


Ist~-D (“from" node 
nee "to" node iD). 





1= idle/busy status 
| 2= current queue size 
{| 3= max queue size thus far) 


Fig. 5.5. SIMSCRIPT arrays (3 Dimensional) 


Another advantage of SIMSCRIPT is the ability to create 
and destroy multivalued variables called temporary entities. 
By using these entities, groups of data can be snuffled and 
processed thru queues relatively easily. Another advantage 
is an efficient utilization of memory space because entities 


which are no longer needed can be destroyed and the memory 


th 


reed for reuse. 

The algorithm in Apvendix & used two temporary entities 
extensively. The first 1s the MESSAGE entity. AS shown ia 
Fig. 5.6 the MESSAGE entity is used for both U-msgs and 


packets. There is more information in the SIMSCRIPT U-asg 
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MESSAGE 


UPDATE PACKsT 
Type 1 2 
Relayer last relaying node 
Next.Stop next receiving node 
Destination Onaag enataong packet's | 
node destination 
nto 1 chanrel session 
value humber 
mrO 2 family of packet seriai 
originator number 
mycO 3 N/A sum of nodes 
hop ped 
Info 4 N/A tine relsased 
frem source 
en f£O 5 group ox Ecaey OF de oun os 
copa ele sbolahy ong non-bdasic group node 





PACK 

Number gueue size due to last change 
entry. Time time queue changed to above size 
Pac.Neighbor neighboring node to which the 


gueue nasS veen changed 


Fig. 5.6. SIMSCRIPT Temporary Entities 


mem in the theoretical 9J-msg of Chapter IV simply for 
Seeereiency of programming and data collection. Note that 
several attributes of the Update MESSAGE and Packet MESSAGE 
have the same meaning and others do not. 

The PACK entity (Fig. 5.6) is used in the CV.LATCH 


° 


routine to calculate ali link channel values. A pack 1 
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reated every time a queue size increases or decreases. 
PACKS are kept ina gueue (called TIME.QUEUE) assigned to 
each node. They are kept until the PACK's ENTRY.TIME (the 
time it entered the queue) is older than the window. sacha 
PACK has a NUMBER which is the new gueue size which caused 
the PACK to be created. Fach node may have several links, 
but all PACK are kept in the same queue. Theretore 
PAC.NEIGHBOR identifies whicn PACKS belong to gach Link for 


a given nede. 


Pee SoLECTION OF TEST PARAMETERS 

For the purpose of the simulation, certain test 
parameters were selected to be fixed and others varied. For 
the fixed parameters, approximations were made based on the 
2stimated performance characteristics of a typical system as 
Meeseribed in Chapter III. fig. 5.7 lists the major systen 
and Simulation parameters. There is no explicit distinction 
between fixed and varying parameters. dowever the estimate 
of 16,000 bps bit rate in Chapter III helped settle on a set 
of processing and transmission times for nessages éestinated 
to range from less than 100 bits (U-msg) to approximately 
1000 bits for packets. The ranges of the varying parameter 


were also affected by the 16 Kbps bit rate estimate on one 





end, and by a performance threshold on the other. These 


results are discussed in greater detail in Chapter VI. 


ee D PRRAMETZRS 


Pkt Processing Time in a Node 
U- “i Processing Time in a Node 
Pkt Transmission Time per Link 
U-msg ace aares™ Time per Link 
yee e))g ee session 

vated ao caels ae eee) 
Links to be samp 
NO. Of Spy les ae sn nk 
Roce iving/t per aneeta ng factors 

for each node) 
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VARYING PARAMETERS 


Period Between Updates 00) ee sec 
Simulation Time Linit __. ; = 2000 sec 
Period Between New Traffic Sessions ~-05 - .1 sec 
Window Size 1 - 20 times 

eee period 
% Inner-Grcup/Family - 75 % 


Pig. 5.7. System Parameters 
Other input date consisted of a description of the test 
Meeeerk (Fig. 5.1) including he identification of nodes, 


groups, families and links. The network topology was fixed 


moc all sinmulaticn runs. 
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Vel erie CONG LU STONS AND RECOMMENDATIONS 


The results and conclusions discussed in this chapter 
primarily involve the simulation of the update portion of 
she protocol in Chapter IV. Furthermor?, in view of the 
wide variety of parameters that cculd have been varied in 
these Simulations, many were fixed at what was considered to 
be reasonable approximations based on performance figures 
used to describe the theoretical system in Chapter III. fhe 
analysis centered around several parameters which were 
considered potentially to have the broadest affect on the 
system response, including the interval between update 
messages (or the rate at which updates were originated), the 
average arrival rate cf new traffic sessions (or at the rate 


a* which packets were created), and window size. 


Be RESULTS AND OBSERVATIONS 

One of the first simulations involved assigning a fixed 
Channel value of 1 to all links in the test network (Fig. 
59.1), defining tne pest path between any two nodes as the 
first path found with the lowest net channel value (also 
Called the shortest path), and then fixing all best paths 


rOr the entire simulation. This is a static routing scheme, 
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using shortest direct paths in the sense of Dinimum number 
ong hops. This fundamental network scheme was used to 
establish a minimum performance level and was used to 
measure the effectiveness of the update program in Appendix 
B. For the network in Fig. 5.1, all best paths were 
calculated and frozen at the beginning of the simulation. 
Then traffic sessions were originated at intervals of .05 
sec and .08 sec. The network quickly congested. At an 
interval of 0.1 sec the network settled down with average 
Sampled queue lengths from 2.2 t0 3.1 for a 2000 sec 
pamglation. 

The remainder of the Simulations involved the update 
algorithm applied to the same network (Fig. 5.1). The tests 
were divided into twe groups. The first and iargest group 
of Ssimulaticns considered the whole network to be one basic 
group (all groups and family ID's were the same). fois is 
essentially unfreezing the static network by applying the 
update algorithm. The second group of Simulations involved 
fhe partitioning of nodes into groups and families which 


could shen be compared to the basic group simulations. 





1. Basic Group Tests 


POimemenem ces:  necwors {Fig. 5.1), the £r0Zen path 
baseline scheme could not function at a new traffic session 
period of less than 0.1 Sec. The basic group tests results 
(Appendix D) suggested that 0.1 sec was also a good limit 
for the Update algoritha. However runs at new traffic 
session periods of .08 séc and .05 séc indicated a graduai 
loss of efficiency indicated by excessive gueue lengths. 
The static network, on tne other naand, had demonstrated 
Catastrophic failure at these intervals. Runs at intervals 
averaging greater than 0.1 sec caused very little strain on 
the update algoritha, therefore the traffic inter-arrival 
interval of 0.1 sec average was selected for the large 
majority of the basic group (and group/family) tests. 

At an average traffic inter-arrival interval of 0.1 
sec, a message (session) averaging 10 packets was added 
tandomly to the network at a rate or 10 messages (Sessions) 


per second. Simulation resuitsS indicated that after 100 


tt, 


rt, 
ib 


Meeanas Of Sinulation ciock tine, any cesidual e cts of 
Starting the Simulation were undetectable. Therefore test 


results were taken for simulation runs varying from 100 to 


1000 sec, Runs greater than 1000 sec gave no indication of 





new information about the length of queues or the rate at 
which packets could be processed. Therefore using an 
average message (Session) arrival time of 0.1 sec and a test 
martation from 100 to 1000 sec, Ehewotinansy, teens of 
Simulations were on the update period (or rate at which new 
update cycles were started) and window size. 

The update period directly reflects th2 amouat of 
overhead required by the networx. ieucre Static RetwOrk, 
+he overhead requirements are minimal, amounting to that 
required *o initially establish the best path network. 
Therefore it is reasonable to expect better performance for 
an incréase in overhead traffic. Tiae) DasiG “Groun. test 
indicated this improvement. 

The average gueue size waS the primary measure of 
performance. The figures derived are conservative 
Caiculations since the sampling in the second half of the 
Bese Was made of the busiest links found in the first haif 
Beecae test, The static network's average gueue length for 
a traffic inter-arzrival interval of 0.1 sec was 2+ packets. 
The basic group aligorithm approaches the static network as 
he Update period approached infinity. Fig. 6.1 shows the 


decrease in average queue size as the traific session 
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The curve labelled "MAX" is the plot of the largest 
average queue sizes cver the set of experiments. 


ieee curveomeebellicd "MIN@SIs ene olot sot the smallest 
average queue sizes over the set cf experiments. 


Fig. 6.1. Queue Size vs Session Intervai (Basic Group) 


interval size decreass¢s. As expected, for relatively long 
update intervals (>1 sec), average queue sizes ranged 
between 1 and 3 packets. From there, as tre update interval 
decreased, average queue sizes dropped guickly. SOURCE Oe | 
sec, the average queue size settled into a range of values 
between approximately .25 and .75 packets. Many runs were 


made in this range and there was no tendancy for results to 





Meeser any pazticular pert of this range as the test 
duration was varied. Results were very stable (see Appendix 


D). 





WINDOW NO LONGER 
DSE® 


Pig. 6.2. Window Calculations 


Window size was also varied to determine its impact 
On average queue size, At first the channel values were 
Calculate over various window sizes based on a Ilinear 
weighted time average. The weighting scheme gave the 
highest weights to the nost recent gueue sizes. However 
Simulation results showed that this scheme resulred in 
larger average queve sizes than a straight unweighted time 


Average as shown in Fig. 6.2 The height of the blocks 
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represent the size of the queve. Their width represents the 
length of time that the queue did not change in size. The 
window indicated how far back on the time line {in Fig. 6.2) 
the program would go t9 calculate the average queue, and 
therefore the channel value for a particular link. 

Window size also proved to be a very stable 
parameter. Window sizes between 1 and 10 times the update 
period gave no indication of influencing the curves shown in 
Fig. 6.1 As the window size increased to over 20 tines the 
update pericd, there were slight increases in average queue 
size. 

The standard deviation of the average queue size for 
Mase Group tests varied slightiy from 1.2 to 2.3 packets, 


with the smallest standard deviation for update periods of 


2. Family/Group Tests 


The basic group test resuits were compared to a 
fundamental static network routing scheme. The family/group 
Sests results (Appendix D) were primarily compared to the 
Maesec Group results. Although tas proposed advantages of 
the family/group aspect of the update algorithm presented ia 


this paper are based on the assumption that the majority of 
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the ‘traffic is confined to inner group or inner family 
eransactions, most of the family/group tests placed no 
restricticns on which node sent orf received traffic. Under 
these conditions, the same set of messages with the sane 
sources and destinations used in the basic group tests were 
used in the fFamily/group tests. Runs involving restricted 
Meanceic is briefly mentioned at the end of this section. 

AS expected, the average gueue size increased in the 
family/group tests. ror update periods ranging from .05 to 
0.5 sec, average queue sizes varied from 9.5 to 1.5 packets. 
Meat ionally, the standard deviation increased tO 
approximately 3.7 to 4.2 packets. The benefits due to this 
drop in performance was the decrease in overhead traffic. 
Typically, the average number of links used by U-msgs as che 
result of a single node originating one update cycle during 
the basic group ‘test ranged from 75 to 90 Ilinks. This is 
because every node aad to have a best path for every other 
node. For a typical family/group test, the average number 
Peeeanxs used dropped to around 23 to 27 links. Therefore 
for a (roughly) 50 percent improvement (decrease) in maxinua 
average queue size, a 200 percent increase in overhead 


traffic was required. 
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Another effect of the family/group algorithm is an 
increase in the average number of nodes hopped by all 
packets. Because of the addressing given to a packet 
Starting out for a nodé in a different family (1t starts out 
with a family best path neighbor), more average links are 
normally required in family/group tests. An interesting 
observation is that occasionally a few (most often 1 or 2 in 
more than 20,000) packets in a family/group test would make 
an unusual number of hops, Cledily Wndicating that it is 
looping due to changes in its best paths. However, 
invariably the total time required for this packet to 
finally reach its destination was weli within the average 
times required by ovther packets meen used far fewer hops. 
The cause of these (relatively raze) Sscllia sons Lsenoct 
obvious. It 1s probably due to the large naumbdsr of links 
meen ClrOSS a group or family boundary. Durlag an update 
each link represents an entry port *o that particular 
related group or family and each acts as an originator of a 
U-msq for that Larger entity (or super-nodés). Therefore 
from update £0 update, the best patn port to that super-node 
could change by a large physical distance. However it is 


not clear if this observation indicates a serious problen, 
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since the packets still invariably arrive in a timely 
fashion. As a safeguard, an additional routine was added to 
Seen Simulation program to check the aop count on all 
outstanding packets when the test ended. At no time was 
there an indication that an undelivered packet was looping 
or making excessive hops. 

Compared to the basic group simulations, relatively 
fewer runs were made for the famlly/group tests. However 
these results suggested that as the ratio of window size to 
session interval increased over the range from ito 10, 
average queue size also increased slightly. Addanionual 
testing may indicate that a relatively flat performance band 
Similar to Fig. 6.1 also exists in this case. 

Finally several runs were made with th2 same test 
network with the additional restriction tnat 50 percent of 
all traffic is inner group and 50 percent of the remaining 
meaeric is inner family. There were too few runs to 
establish a trend. However the results suggested a decrease 


in average gueue size and standard deviation. 


SB. CONCLUSIONS 
This was very much a preliminary investigation of this 


network management protocol. It would be improper to 
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identify much more than broad performance characteristics or 
trends. 

The update algorithm clearly functions properly. Both 
the basic group concept and the family/group concept 
responds *o changing channel values and provides routes that 
can be used under a reasonable traffic load. The aigoritha 
is also very stable and robust. Fig. 6.1 indicates that for 
a traffic session interval of 0.1 sec and longer, the 
algorithm has a gcod and very flat performance curve for an 
update period of approximately 0.1 sec and less. 

Investigating the update intervait should continue to be 
muetecal DOiNt in future analysis. For a given network 
performance level it wili always be important <to minimize 
the update interval, since 1%t reflects the overhead traffic 
that the network must process. 


Ga the other hand, as new and proken links are 


ct 


integrated sitio the Simulation, they Want) add 
counter-arguments to the continued increase in the update 
periods. AS licks are broker, 2z alternate links are not 
available, nodes rely on U-msgs to unfreeze traffic and 


provide new best paths. Therefore future analysis must find 


a balance that will optimize this situation. 
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The family/group concept provides very good economy (and 
robustness, as compared to gateway nodes) with a modest 
decrease in performance. Prope networks which operate 
extensively within smaller sub-network boundaries (Such as a 
typical military network), the family/group concept appears 
to offer a significant savings in cverhead traffic compared 


to the same network operating as one basic group. 


Peet SCOMMENDATIONS FOR FURTHER STUDY 

This preliminary study indicates that the update portion 
of the decentralized routing protocol described in Chapter 
LV accomplishes the fundamental requirement of routing user 
femectic. Follow-on investigations are needed to integrate 
the new and oroken link concepts described in Chapter IV. 
Wherever possible, the program in Appendix E was written to 
Me@Gilitate this next step in the investigation. It can 
Similate the loss, gain, and the planned movement of nodes 
by scheduling the failure and awakening of links on the 
eemulation clock. Both single node and group novements 
could be simulated. 

This protocol was conceived to be simple and practical. 
A degree of simplicity has been retained. However to become 
a practical protocol, there are several topics which need to 
be considered, that are not addressed in this study. 
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The primary problem is how to cope with splintering. 
Splintering may be a single node which crosses a group or 
family boundary. If this problem is applied to a single 
basic group network, it then reduces to the problem of a 
node leaving the network or a new node entering the network. 

Splintering may also be defined as groups of nodes being 
completely cut off from the other nodes in its basic group. 
This sub-group may be left to operate autonomously or find 
itself in the middle of another basic group. AS eiae cuca. 
example based on the Military organizations mentioned 


earlier would be the movement of a company, which is part of 


a battalion's basic group net, thru another battaiion's 
SaetOr. It is conceivable that the company may iose ali 
direct links with its basic group during the movement. A 


practical protoccl must alse accommodate this splintering to 
the point where a basic group is divided into two or more 
equal parts, leading to the guestion of determining which 
Group is the splinter and which is the remainder of the 
Seaginal basic group. 

finally, asSu@ing 2 practical protocol can be ftuliy 
developed, there is the much more difficult problem of 
measuring its efficiency both in an absclute sense, and in 


meaelOn to Other existing decentralized algerithms. 
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APPENDIX A 





ALTERNATE LINK THEORY 

CLAIM: During a shortest path update process, it may be 
possible to identify an alternate link which may be used to 
Maintain traffic flow (although at some degraded level) in 
ehe event that a node's IMMEDIATE DOWNSTREAM link in a 
particular shortest path is broken. Tae, Swaeeh wall result 
in a non-optimum but LOOPFREE network. LOOPPREE ila tas 
discussion implies locpfree in the narrow sense. That is, 
traffic leaving anode for a given destination is assured 
Brat it will not icop back into tae sending node. 

DISCUSSION: Simply stated, the concept 1s that some 
Optimum path routing algorithms nay acquire information that 
is normally discarded, put May de used at individual node 
Maer tO Switch traffic to an alternate link if a break is 


found in that node's immediate downstream link along the 


ct 


b 


@ 


optimum path. This switch does not necessarily i¢ave 
rest of the netwerk optimally routed, but it is LOOPFREE. 
I+ may be useful as a temporary f£1x untii the next update 
process is received. 

PROOF: Any set of shortest path routes in a network can 


ne expressed as a spanning tree, which is always loopéfree. 


125 





In the following examples, the Spanning tree is vertically 
scaled to represent the channel value/distance to the tree 


root. Every link is assumed to nave a Minimum value of 1, 
however this is not critical in the proof. 

A loop implies that a route passes through the same node 
More than once. 

If every link has a niainum vaiue of 1, then the total 
distance for each node in an optimum spanning tree to the 
root must be at least one larger than the next aode 
downstream. 

Therefore a loop cannot exist in a Spanning tree because 
once traffic leaves a node in a Spanning tree, it will never 
arrive at anode of equal distance to the root. Under 
normal operations, if more than one node has the distance to 
mee root cof d, then a particular message for that root wili 
Only pass *hrough (at most) one of thése nodes of distance 
om Furthermore, once traffic reaches a node of distance 
less than d, it will never pass thru any node of distance d 
under normal traffic flow conditions. 

Consider the following network and spanning tree with A 


as the root. 
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Applying the concept that a message will only pass thru 
one node of distance d to the above spanning tree, we see 
+*hat if a message is 4 units away from node A, it is EITHER 
at node D,E or F. The message will be at one of these three 
and will never pass thru the other two. 

Similarly, if a message at node G was transplanted in 
node D or a message in node 9 was transpianted in node C 
(where C and D both have distances 13ss than # andG) the 
message could continue to the root without ever passing Dack 
mead the initial node {HH sor SG). Note that this is not the 
case if a message in aode C was transpianted in node H. 


Meme aiso that node H's distance is greater than node C's 
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d2astance to the root. iis 25 ua0 andicator that a Loop 
jondi tion 1s possible. 

Reviewing the above network diagram we see that there 
are many unused links if traffic to Ais restricted to the 
Best Paths (—>—). However if an established best path is 
broken, these links provide a potential bridge which may 
transplant traffic from the node wnich discovers a broken 
link, +o an adequate adjacent node which circumvents the 
beoken link. The necessary and sufficient factor to 
determine whether an adjacent node is adequate is its 
Mestcance £0 the root A. As long as the adjacent node's 
distance to Ais less than ar equai to the distance via the 
broken Best Path link, traffic transferred to 1t will never 
loop back to the transferring node (and presumably vill 
proceed to node A). 

APPLICATION: Consider a Simple algorithn where 
cummulative distance to the root/sink was used to determine 
the Best Path. At each node, one best path would ultimately 
be selected over cthers. In making this seiection each aode 
learns the best path distance to tne sinx thru its adjacent 
neighbors, adds its distance to each adjacent neighbor and 


Peeks <he 8est Path. However the pest path distance froa 
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the non-selected adjacent neighbors may still be useful. If 
+his distance is <= the node's best path distance to the 
sink, and if the node detects a break in its best path link, 
i+ may transfer traffic around the broken link by using one 
of the adjacent nodes, with the assurance that the new path 
is loop frees. It is perfectly conceivable that more than 
one alternate link may be available to a given node at one 


time. Using the network andthe tree shown above as an 


example, we have: 


; De / POSSIBLE ALTERNATE LINK(S) / 

{distance *o A) (distances) 

B/ (1) None 

C7 (2 None 

D/ (4 E/ (4) 

E/ (4 CA D/ (4) 

F/ (4 None 

G/({7 D/ ai E/ (4 

H/ (5 Cae3) , ott 

I/ (6 None 


ieee that the tzansferring node aust still addthe link 
distances from it to the selected adjacent node pbefore 


Dicking the shortest alternate path. 
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APPENDIX B 





DISTRIBUTED PROTOCOL ALGORITHM 


ARAKCMHLELLESANARETARRSTLSETRALRSEAS EHAHHAAREAPASAAAAAELSKEHHSCESEHEH ERASE ESSE AESEE F 
SYMBOLS AND DEFINITIONS , 


A(d) ==> Set of alternate neighbor nodes for destination d 
L(d) ==> Distance from 3(d) to.d (G/d or F/d)* on best path 
AL(3,1) ==> Distance from A(d) =,1 to d (G/d or F/d)* on ls best ee 
AX(d) ==> Message to neiahbor indicating it nas been selected 8(d) 
B(d) ==> Neighboring node on best path to d (G/d or PF/d)* 


* (3/d or F/d) indicates that Rees or fasily identities can be 
usé in the argument of these synabols in place of d to define 
intra-group and intra-family operations. 


G/n ==> Beene identity of node a 
_P/n ==> Farily_ identity of node a ‘ ; ; 
4(1,0) ==> Channel value (distance) of link from sode i to node a 
D(n}) ==> Distance from node n to node don its best path 
d* ==> d, G/d_ or ?/d for inner group, liatra-group or 
Latra-family operations as required ; ; 
TRANS ==> Originate or relay a message to appropriate neighbor(s) 
EXIT ==> No further acticn required 


© FREKREKTAREAKTARALKELESTKKCLAERCKARKAST ENARCAREKLAKECKSEKRAKSCSESSTKSETSESESEHEKRTREKEKEKEVESESES 


A. UPDATE INITIATION 
Ti. Xmt O(1,d,D(1)) to all neighbors where 
D(1)= 0 
II. Schedule next Update origination in designated Update period. 
“8. 8&SCEIVE / PSOCESS UPDETE 
I. 3204 U(1,d,D(1)} at node i, & G/l = G/Yt 
1) G/i ne (not equal) G/1 
Etat 


2) 3 8 (4) = 1 —s 
Liejis= 501); BAe D(1)+ d(i,1l) 
TRANS O(2,D5D (2) 
EXIT ; 
3) If Bid 1 & D(l)+ a (4,1) <= DL 
rf ta} < a(ays 4 oo5 (2) 
a(dyzs a (a) 0 3 (a) 
AL (d,3(d)):= L(d) 
lfeaay Le aol 
a (Nee a(d)-1 
Lidj:= Dl); D(ips= D(L)+ ail 
Ua i= 24y; piye= D+ asd) 
2XIT 
8) If B(a 1 6 D(l) <= D(i 
) NORE ete! ' (1) (1) 
AL(d,l):= D(1) 
EXIT 
5) Tf D1) > 04) 8 Le a(a) 
eyie tO) 
II. RCV U(L,D,D(L)) AT NODE I & G/I NE G/D & P/I = P/D 


ise 





PILE: APS PINAL A NAVAL POSTGRADUATE SCHOOL 


1) Tf F/i ne F/1L 
EXIT 


2) TE eee = G/d & 1 ne @ 


3) B =} 
22 D D(G/D) := D(L)* D(I,L 
Beg a(i, Soh nt /a) (1) sod 
O) LEeroa) + a(t, 1 = D(G/d) & B(G/d) ne l 
2092 20 < D (lis hase hy SS il 
BiG/d) 32 A (a) u B (G74) 
AL (6/448 {(G dj )2= L(G/a) 
If le erd ) 
A(G/d) := A(G/d)- 1 
Piet i t,o ceyay ze oye aden 
TRANS U(i,G/d,D (G/4) ) 
EXIT. : 
5) ff 3(G/d) ne l & D(2) <= D(G/a) 
A(G/a) := A( AN 
AL(G/d,1):= D(1) 
EXIT 
6) If D(1) > D(G/a) & aA(G/a)= 1 
Bagh ea toe ae 


IIE. RCV O(L,D,D(L)) AT YODE I & F/I NE P/D 
Lt ae & 9 ae = F/d &6 1 ned 


Eat 
a Page ="y{L) P/D) s= O(L)* D(I,L) 
taal fs’ Gi4, ain 7a) ; ’ 
S)) Lee Oly? d(iy = D(P/d) & B(P/d) ne l 
| apa TN all - 
LP /4, oH Seva = L(F/d) 
Lee A(F 74 
A(P/d):= A(P/d)- 1 
ae 5 (2) 3 D(?/d):= D(l)+ d(i,l) : 
TRANS Wey; d,D (374 yy? : 
op GB ist 
4) If B(P/d) ne & D(1) <= D(P/d 
) TE EG ME bh Of eZ 
AL{(F/d,1):= D(1) 
EXIT 
5) If D{l) > D(P/4d) any A(P/d)= 1 
Seac i= Mery < 1 
EXIT 
*raffic frozen due to X(d’') is released as soon as a new 


Btn") is selected. 


oe) SROREN LINK PROCESS 
z. Node 1 discovers link with node j is broken. 
For each d* s.t. B(d) = j : 


+ 
OO 
}— 
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1) Te Aa") ne. 0 ; 

nz= A(d*) for which AL(A(d"),d*)* d(i,a(d*)) 
1S min over A(d*) 
S:= AL(no,da°*) 
B (a 22 oO 
Lid*):= 5s 
TRANS AX(d*) to a 
EXTLe 
~ 

2) If A(d*) = 0 ; 
TRANS X(d") to all neighbors 
Preeze traffic destined for d' 
B(d*) = 0 
Lédt = oa 
EXIT 

II. Rev cal at node i from node 1] 

v1) Lt. 3G") = 
PARA. I.1) & 2) ABOVE 

2) rf Bcd?) nel 
EX1T- 


III. Rev AX(d*) at node i from node 1 


rf any A(d*) = 1 
A(d*):= A(d*")- 
exry oa - 





APPENDIX C 


WORST CASE GROWTH Of UPDATE MESSAGES 


ims untakely that any distributed routing algoritha, 
Which attempts +o balance traffic on ali links in a busy 
network and takes a finite time to update a network, will 
ever produce a network-wide routing scheme whico is truly 
optimized. For example, delays due to propagation time and 
processing time in each node will cause a time difference 
between the node which originated the update and che last 
node which was affected by that origination. Disangs thas 
memod, PDarcicularly under heavy traffic loads, conditions 
which existed in and around the originating node at the time 
the update was started may be very different from those 
existing by tne time the nost istant node is updated. 
Generally, Pac tenGer “ie cakes L£Or an” Update cycie to 
propaga*e throughout the petwork, and the longer the tine 
between update cycles, the more conditions could change, 
thereby degrading an ideal routing scheme. Thezetore. it is 
Memiene = consider hew long it would take for an algorithno 
momOpt.Mmize anetwork if a sudden change in link status 


Caused a worst case Situation. 


Li Sh: 





Bey. 2S) eportant £0 understand that the following 
analysis assumes that aiter the status (e.g. loading) of the 
links have changed, they are theoretically frozen untii tne 
network achieves re-optimization. (cnOtee cas assudpt. On, 
as stated above, it may be impossible to arrive at a rully 
optimized routing scheme in a changing network at any point 
in time. 

For the algorithm presented in this paper, under worst 
Sase conditions, it could take up to (approximately) 
3X (n®* 3) update messages to optimize a network of n nodes. 


The following figure shows a richly connected network. 





# Indicates CV to neighbor 


# Indicates net CV to Node 1 
on Best Path 





(+) Indicates new CV at next update 
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Network A shows the best paths to Node 1 as of the Last 
update. To maintain the worst case conditions, we will 
assume that the channel values of the links inside the 
network (the star) are always so large that they wiii never 
be selected as a best path iink to Node 1. However aach of 
hese links iii require that another U-msg be sent each 
time a node at either end updates its current dest path or 
adopts a new one. 

The channel values in parenthesis in network B represent 
changes in the channel values since the last undate and will 
be used in the next update cycle originated by node 1. AS 
miencitst update cycle begins, all nodes receive a U-msg 
meom noade | (fOr n=5 nodes, a-! U-msgs are initially sent 
out at the beginning of a cycle). It is assumed that (under 
worst casé Gendi ton s) che U-msg is relayed 
counter-clockwise (CCW) thru node 2 upstream along the best 
path arriving at node 5 sometime after nodes 3,4 and 5 have 
rejected the direct U-musg from node 1 (because they compared 
Their outdated net best path channel value to the current 
channel values in the U-msgs.). 


As the U-msg thru nede 2 works itself upstream along the 


best path, it updates each nede's net best path channel 
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value and causes each node to relay the update to all 
neighbors except the sending node (n-2 relays for n-1 nodes) 
Soausing a relay of (n-1)(n-2) U-msgs. Adagag tse oOliginal 


n-1 U-msgs from nede 1; 
Ceape(n-2jeetetn- 1) = (2-1) (a=2'1) = (2-1) #8 2. 


In this first update cycle, (n-1)%*2 U-msgs have gone out 
and not a single node has changed its best path neighbor to 
node 1. But this was still a significant step because each 
node now knows the true distance along its best path to z2ode 


1 as shown in Network C. 





eetinadicates net CV to Node L 
on Best Path 


Some time later node 1 originates its next U-msg cycle. 


This tine (worst case) it is assumed that the U-msg thru 
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node 2 works its way CCW to node 5 before node 5 receives 
its U-msg direct from node 1. This is nearly a repeat of 
the previous cycle causing another (n-1)#%2 U-msg to be 
initiated. However when the U-msg direct from node 1 
finally arrives at node 5, node 5 picks anew best path 
(direct to node 1) and relays the U-msg to ail neighbors 
(except the sending node). When node 4 réceives node 5's 
U-msg, it selects this new best path, informs all neighbors 
and the process continues until the network has now selected 
the optimum routing scheme as shown in Network D. 

mets final series of U-msgs involving (n-1) (2-2) 
z~ransmissions prings the total U-msgs generated over the two 


update cycles (originated by node 1) to 


ae 1 eee + (in - 1) (a= 2) 


which ls apvoroximnatealy 


3((n-1)422) --> 3 (n**2). 


Since there are n nodeS in the network, each initiating 
17S own update during a single network update cycie, the 
total (worst case) number of U-msgs possible 1s 3(n**3) 


(over two network-wide update cycles in this example). 
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APPENDIX D 
SEMULATZION KkESULTS 

This appendix illustrates many of the results of the 
Simulation runs for the program in Appendix &. For vat! 
results in this annex, the oniy parameters varied were the 
Update period, window size and time limit/duration of tae 
Simulation run. Data is divided into two major areas; basic 
group test results (3G) and family/group test results (F/G). 
Each plot corresponds to the Update period size indicated to 
mee Lert. The plotted data is the av2rage gueue size for a 
run derived from sampling 10 iinks (those having the highest 
average queues over the first half of the simulation rua) 
approximately 1000 times each during the second halt of the 
Sebati On run. Results for a given test duration are 


represented by a number corresponding to the size listed 


adjacent to the plot. The vertical axis is average queue 
size. The horizontal axis is the Window/Update Period 
Tat 1.0. 
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APPENDIX E 
NAVAL POSTGRADUATE SCHOOL 
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PILE: DOC s A NAVAL POSTGRADUATE SCHOOL 


=) 


PCTOR AS REAL 
MAX.U.HOPS aS 


v 
v 
E 
SMP.CNTR AS V 
§p UPDATE AS RE 
IABLE 


pe 
ze OH 


=Heywcw 


aauaicuce 
ARIABLES 
ARIAS 
AL VA 


R 
L 


we eSinhitimininty 
Om May day giydg 
wrerers Te terol | 
ese Se SLLZAzs 
oricniintirsts 
SU Cig tHiro 
te & Hino 
rola e<ro'ge 
OOS ck trte 
td 
oe tgkto? © 


| 
tae 


Aes SET SOME VARIABLES 


é SET THE RANGE OF PACKETS PER SESSION (PKT.MIN 4 MAX) 
Pecer oA LEmi® ON THE NUMBER OF SESSIONS (TRAFP.LI aN 

€ SET AN UPDATE MSG TO PACKET PROCESSING RATIO (U.PAC.RATIO) 
7 ET AN UPDATE SSG TERMINATION TISE (U.XHN. TIME) 


S) 
K 
K 
S 
E 


R QCH'ov 


AS POLLOWS 
NIT REGEILV 
TOR PACTO 
ODE 


~wrdnde# ertttrtrtpt ow ow ew oe we weg o *ygUUUUO0DVUD 


E 
R 


OwzZzowty - © etymWtitmty «© e©<e«seeas ao 


ma 


SMIT.PERCENT(NODE), RECSIVE.PERCENT (NODE) , GROUP(NODE), 


CEIVE VEACTO 
ii GROUP NUMBERS. 
T GROUP NUS 


W>ty 


N.NODE, DO 

YT = T&NS.PCNT + TRANSMIT. PERCENT (I) ae 
= RCV.PCNT + RECEIVE. PERCENT (I) 

OUP (I) 

GROUP (I) 


ee 
Me ¢ SET PROGRAM GRP NUS 


¢ 
LE? GROUP(I) = GROUP(I) + N.NODE 
S22VE FAM.OP.GRP (*) AS. (GRPS + N.NODB + 25) 
R I = 1 TO N.NODE, DO 
IP PMLYS < FAMILY (T) 
LET PMLYS = PAMILY(D) 
REGARDLESS r 


@ SET PROGRAM Fae NOM 


Mao ece ="N.NODE + GRPS + Ff 
AM.OF.GRP (GROUP(I)) = FAMILY 


S = N.NODE + GRPS + PMLYS 
TO N.NOD2, DO 
LINZ WITH 1, TRANSMIT. PERCENT (I), R2CBIVE. PERCENT (T) 

es N.NODES, GROUP(I), (PAMILY(£) ~ N.NODZ - GRPS),° FAMILY (I) 


aa xm EH RE, £2XR az (#8) ¥# (8) 


LOOP 
SKIP 1 OUTPOT LINE 


AMILY (I 
(Hy (I) 
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be ve oe ae 


Ge'giwpy 2«e« ete ene eaece2e2ane 2 «2 aQuvuywy 
HANI Ics UOOU 


Bear Oro etnimtpy - 


R HW NYT NNOWO'R HH, 


fr 


moti 


¢ 
eg mutt yig't'g © = eM Pins vUVGa 


“HM MmMtiolhd © = eRe 
NMAMNAIANMIN 


LUtIOIL 
939 FU 0 oa 0 a og 


my 
f=] 
rH 
i> w OTORO 
(+) De AH 


ig 


"ZZ 
eae 
‘ge 
'¢¢ 
*¢¢ 


aeevwraee wf4t a 


e 


DOC Ss A NAVAL POSTGRADUATE SCHOOL 


Zt Er Or 
Or me ANU 


P MEANS THE PERCENTAGE OF GENERATED TRAPFIC THAT WILL NOT 
ITS BASIC GROUP. SIMILARLY FOR IN.FPAMSILY. 


ate Sage ae oe THE LEVEL OF DIAGNOSTIC PRINTING. 
ALL PACKETS + LISTS INITIAL NEIGHBORS + LIST 
M7 2CY OAT Bo OF RUN 


“0 OM pti 


Ga 
MH MO Citditm Hey 


a?) 
2 
ar 


WN OG me 
ae | 
Vv 


-> 
SMP. LINKS I 
NO.OP. SAMPLES 

AT PACH OF SMP.L 
LINKS IS THE TOTA 
IN.GROUP, IN. FAMILY 


PRN 
SMP.LINKS, NO.OF.SAMPLES 


: 
By 
ak 
st 
A 
fi 
2 ER OF LINKS TO BE SAMPLED 
E 135 ieee a MEAN NUMBER OF SAMPLES TO BE TAKES 


oe. 
COINKS ON @HE CAST WAMF OF TEST. 
BOR OF LINKS IN THE NETWORK. 


LINKS 
8 LINES WITH UP.DATE.PERIOD, PROCESSING.TIME, PKT.XMN. TIME, 
.LIMIT, TRAP. LIMIT, AVE. NEW. TRAFFIC. INTERVAL, WINDOW, 
MIN, PKT.MAX, IN.GROUP, IN.FAMILY AS FOLLOWS 
B PERIOD IS #&el.axexeeun SEC 
SSING TIME IN EACH NODF POR ANY PACKET IS .*#8@a* SEC . 
T TRANSIT TINE SETHEEN ANY TWO NODES IS .s#eee® SE 
DURATION IS #2", SEC. TEST LIMITED TO *##e% TRAPPIC SESSIONS. 
RAFFIC SESSIONS. ARE. STAETED AT AN AVERAGE INTERVAL OF *%.#29¢8% SEC 
ZL VALUE CALCULATION WINDOW IS +#®,a=eae8 SEC 
TRAPPIC SESSION VARIES FROM *2 TO &e PACKETS 
AST **. % OF TRAFFIC IS INNER GROUP, ANOTHER ®*. % IS INNER PAMILY. 
1 OUTPUT LINE 
SZ2EB CHAPTER 6 FOR DESCRIPTION OF ARRAYS 2 
VE LINK.ABLE(*,7) AS LINKS BY 6 
VE LNK.MONITOR(*,*,7) A .NOD2 BY N.NODE BY 3 
VE HOP.COUNT (# ae AS (ge 4 NODE) 8 2 roy 
V= CLOCK. DATA (# AS “(U2 NOD ) ty 2 
ST.MAX.OF. TRAE ofc) = 2 INP. P(TIMESLIMIT / AVE.NEW. TRAFFIC. INTERVAL) 
VE TRACER (Z,%) AS Esé. MAX.OF.TRAFPIC BY 2 
VE SMP.SET a.) AS SMP.LINKS BY 2 
VE QU.DISTH(*) aS 250 - 
1 LINE AS POLLOWS 
NK 


TO LINKS 
1 TO READ LINK. ABLE(I,J) 
" 


LINE WITH LINK.ABLE(I,1), LINK.ABLE(I,2) 4S FOLLOWS 


ae SCHEDULING INITIAL ZVENTS 


PenecvULE THs 2IRSL GOPDATE FOR EACH NODE ee oh PO SkCE) 
megeoo ALL G¥%S FOR THIS JPDATE (C¥.LaTCH) 

SEenEOULE FIRST PACKET SESSION 

SCHEDULE TEST FOR MAX QUEUES HALF WAY THRU TEST 


POR EACH NODE 


GeO. i A 
(0.0 a1 


ai eUPDATE.ZESSAGE GIVEN NODE, UPDATE IN (UNIPORM. ? 


}) ONITS 
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SGHEDOLE A® CY.GATCH AT 0.00 
SCHEDULE A STOP.SINULATION IN TIME 
SCHEDULE A NEW. eee eae GIVE 
¢ EXPONENTIAL. P(AVE.NEW.TRAFPIC.L 
SCHEDULE A ey PLER TN Ge M 
SERVE BEST. PATH (*,7%,%) AS N 
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€ IDENTIFY NEIGHBORS, S 


1 TO N.NODE 
EST. PATH (I,t, 1) = fr 


NEIGHBOR.LIST(#,*,*) AS N.NODE BY 6 BY 3 
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EIGHBOR.LIST LINK.A 
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JEIGHSOR. wien LINK. ABLE I,2) 0d 
LET NPIGHBOR. L LINK.A an +2] 
LET NEIGHBORS. ret Disk, ABLES (lL, 2 
GO LAST.STEP 

ELSE 


LOOP 
‘LAST. = Ae 
LET NET 


md 


UT LINES 

E AS FOLLOWS 

ND CHANNEL VALUES 
NeNODE, DO 
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wdIisry'Yy 
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OUTP N 

Ne WITH I AS FOLLOWS 
IGHBORS AND CV 
i 6, 


OOwsrt 


fe) 
LINE’ WITH NEIGHBOR.LIST(I,J,1) AND NEIGHBOR.LIST (I,J 3) 


am a2 


LOO 
REGARDLESS 


''¢¢e¢e 
tte e SIMULATION 


tt¢g 

t 8 

END ‘tOP MAIN 

t¢ 

‘'¢¢ THIS ROOTINE HALTS THE PROGRAM AND GIVES SEVERAL STATISTICAL 
eee seeOnts ON THE STATUS OF THE SISULATION. AFTER FOUR REPORTS 
re THE ROUTINE STOPS THE SIMULATION. 

Event STOP.SIMNOULATION 

Beet c TOL nOPS,)l0T. PACKETS anvD DELIVERED AS VARIABLES 

ago oy oe oe AND AVE SNODSS. HOPPSD AS REAL VwRIABLES 

BPESSis RATIO AND IDEAL. TINE AS REAL VARITASLES 

DETINE Y AS A REAL VARIABLE 

DEFINE SD,XR,22,22S04% AS REAL VARIABLES 

(tes DEEK COUNT THE FOUR REPORTS 

moce TOTSPACNETS SUNS THE TOTAL PKTS GENERATED OP TO THIS POINT 
‘*gz DELIVERED SUMS THE TOTAL PKTS REACHING THEIR DESTINATION 
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ADE BY ALL PKTS 


SAMPLES 
LE SIZES SQUARED 


Wend 


hm 
rg mghi x 


BR AAANHAHAY RRAA 

em Te Tao | 

eyOus 

raters 

© rie KR 

NW dro il 
Om» 
Oornrwonr 


¢ ORINT BP NEIGHBORS AND C¥ 


UH © © seo ott ee eee 
On oo =*(KCMlo1olinito oo ea @ 


pa re 


P E 
INE WITH I, AS_ FOLLOWS 
EAee NODE ** at -~- DESTINATION - BP.NEIGHBOR - CY FRS BP.NODE 


Bs (ee QUP {I ie = 60P | ait AND eat 
LINE WITH EST. PATH (Ig Jil). BEST. PATA (Iss 52) AS Eoueous 


N.NODE+1) TO NGP 


DO 

UP Beda NE 6 AND LY(I) NE J AND J > N.NODE} 
a an 

M 

6 


4 
AMILY(I) AND J <= (N.NODE * GRPS)) 


3g 
0 
M 


and He: 


LINE WITH J, BEST.PATH (I,J,1), BEST.PATH(I,J,2) AS POLLOWS 
me sx 22 


Oo 
c= 
an | 
lar | 
ty 
Gityru Rd uN C4 


Me 
Crpry e 
Pow WwWHNOoO 


Or 
va 
© 
a ® 


GARDLESS 
¢ COUNT PKTS CREATED AND DELIVERED 
1 TO TOT.NEW. TRAFFIC, DO —~-. 


OT. PACKETS = TOT.PACKETS + TRACER (I,1) 
ELIVERED = DELIVERED + TRACES (2,2) 


'g 2 @ ersygt* 


= 
™m 
od ~ 
D 


¢¢ PRINT SELECTED INPUT DATA 


IN REPORT ON A NEW PAGE 
LINES WITH UP.DATE. 
TRAR-LENTT AY 
-MAX, Iv. Gkoop 
= vee ke we 

PACH NOD 
E BETWEEN 
nm rtrkrtn 


k 
M 
te 
ONS ARE Ss?! 
& 
L 


PROCESSING.TIME, PKT.XMN.TIME, 
AFFIC. INTERVAL, a#INDOW, 
ILY AS FOLLGHS’ 


NY PACKET [S .S#@888 SEC 
TaO NODES FS .Meeeee SEC 
TeESt LISITED TO eetee: TRAFFIC SESSTO 


D AT AN AVERAGE INTERVAL OP *#, 28 
W IS xe auekawe STC 
N 


{wR TO Re PACKETS 
ER GROUP, ANOTHER ee. % TS INNER PANTILY. 


1 4 


wy 


c 
S 
E 


t4lfeje bird 


‘Qiu r J HYU ZlH 


NS. 
SEC 


nein ela Lene | 


ULATION aI 
OW VARIES 
PeUsAce LC TS Ph 


OF BNANODGAHHOD 
Os, FINO ew 
Wurst MQ eit 

rg <g og d 
e'O2HHS OD KH 
RHEOWM Wot) 
© OQ OMNIOAS 

a 
= 

gmap Py 

Worry aWgNnHwssy 
@aO0Onuna«Koa 


8] 
O HH rin 
ar 


N) 
ca 


ty owed 
WH Berg tCJusryUrdice 


a 


hr oF 


tHitdwr) 
Oo 
Onv wg 


INE 

AS POLLOWS 

° MEAN TIME PEAK TINE IDEAL 
S PER PKT TIME TIME 
ire ooh 


DO 
(Ls SACK eS) YE 0 


OOOWRN @ © af{Ve ey EP uUPy nil © @ aAO 
Owed 


YO) = @ weds od 


HUeDVORHH A 
© = 


| 
ta 
© il 
ro 
2) 
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HOPS + fr, vi HOP. COUNT (I, PACKET) } 

CK. DATA (I, 1) / BEAL, Fg OP. COUNT (I, PACKET) ) 

ME = IMPKT.XSN. TINE ¢ (1-1) #PROCESSIi ~ CIME 

WITH igus HOP. COUNT (I, PACKET), AVE.TIME, CLOCK.DATA(L,2), 


= Feeene *  saeees & eeeeee 


lee PRINT ALERT SSG IF A PKT HOPPED MORE THAN TOTAL NUMBER CP NODES 


SG.HLT NE O 
SKIP 2 OUTPUT LINES 
PRINT 1 ae AS FOLLOWS 


===== NOTE AT LEAST 1 PACKET HOPPED MORE THAN THE TOTAL NUMBER OF NODES = 
SKIP 2 OUTPUT LINES 
PZGARDLESS 
let AVE.NODES.HOPPED = REAL. F(TOT.HOPS) / REAL. P(DELIVERED) 
lee PRINT SELECTED STATISTICAL DATA 
SkI? 3 OUTPOT LINES ~ 
P=INT 1 LINE WITH AVE.NODES.HOPPED AS FOLLOWS 
MZAN NUMBER OF NODES HOPPED PER PACKET IS *8,# 
ee lP ' OUTPUT LINE 
crootn OF LINE WITH TOTSNEW.TRAFFIC AND TOT.PACKETS aS FOLLOWS 
A TOTAL OF REN NEW XMNS WERE STARTED (TOTALING 8888 PACKETS }. 
Seon. Vl LINESMITH (TOT. PACKETS - DELIVERED) AS FOLLOWS 
OF Talos. mem PACKETS WERE UNDELIVERED WHEN THE TEST WAS ENDED. 
eee) OUTPUT LINE 
Pee SATIO = REAL.F (NET. SO OA a a a a 
PFINT 1 LINE WITH RATIO AS FOLLOWS i. 
FCR EACH NEW UPDATE, AN AVERAGE OF #*",8® LINKS WERE USED. 
etre? OUTPUT LINE 
PTINT 1 LINE WITH SAX.U.HOPS AS FOLLOWS 
LVIGEST BEST PATH AT ANY TIME WAS *®= LINKS. 
‘tee SKI? TO END OF ROUTINE IP STILL IN FIRST HALF OF TEST 
TP Peon < 3 x 
GO CON.TINOE . 
P1SE ‘ 
tee PRINT THE NUMBER OF LINKS SASPLED AND TOTAL SAMPLES PER LINK __ 
ciID? 1 OUTPUT LINE 
Beers LINES WITH SMP. LINKS, SEP.CNTE AS FOLLONS 
OSaUE LENGTH DISTRIBUTION 
fo LLNKS WERE SAMPLED 
max SAMPLES / LINK WERE TAKEN 
END **OP BACKROUND DATA 


ae PRINT MAX Q LENGTH 
IF PRNT >= 0 


Shae 2 OUTPUT LINES 
DSRINT 2 LINES AS FOLLOWS 
PaXIMUM QUEUE LENGTH: 
FRON TO MAX 
FOR, IT = 1 TO LINKS ¢ DO 
LET A = CENK. ABLE fe 12) 
Pore oe = LINK. ABLE (1,2 
PRINT i“. LINE itu A,3,LNK. SONITOR(A,B,3) AS POLLOWS 
PRINT te ae aoa 3,A,LNK. NONITOR(5,A,3) AS FOLLOWS 
LOOP 
S2GARDLESS 
''ee PRINT THE SAMPLING COUNT OF Q SIZES FROM 0 THRO 250, AND COMPUTE 
*'gé@ SUMS TO CALCULATE AVERAGE AND STANDARD DEVIATION 
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ee 

BEGIN REPORT ON A NEW PAGE 

Parva | LINE AS #OLLOWS 
Q-SIZE - SAMPLE DENSITY 


FcR I = 170 250, DO 
(I-1f*QU.DISTR(I) + SOM 


LET SUM = 
LET X=X+QU.DISTR : . 
EET Z2=. OU. DESTR(E paces . 
LET Z2SUS%= 72 SUH+ 
IP QU. DISTRI NE 
PRINT 1 LIN NETH (I-1) AND QU.DISTR(I) AS FOLLOWS 
mw ess 
REGARDLESS 
LOOP 
L=7 Y = SOM/X 
PEINT 1 LINE wITA Y AS FOLLOWS 
AVERAGE Q LENGTH = ee 
LET SD= SORT.F((Z2SUM= XIR®(Y®"2))/(£R-1.)) 
S<Ip 1 OUTPUT {fe 
SS™NT 1 LINE aren. SD AS POLLOWS 
S“ANDARD DEVIATION = #%,$88 : 
‘'¢¢ CHECK ALL UNDELIVERED PXTS. REPORT ANY WITH A HOP COUNT > N.NODB 
T=INT 1 LINE AS POLLO@S 
GNUSUAL DELAYS POR PACKETS NOT DELIVERED DESCRIBED BELOW 
P°2 NO.DE = 1T0 N.NOD%, DO 
202 FACH MESSAGE IN guEbz (xo (NO.DE) WITH TYPE(SESSAGE) = PACKET, DO 
=F NODES, HOPPED (MESSAGE) >= N.NODE 
PRINT 1 LINE ITH RELEASE. Tins (MESSAGE) , NODES. Bon) Gage) | 
Ss) £5 
PACKET RELZASED AT **® eee@ee AND HAS €*®* HOPS 
22GARDLESS 
LCOP 
LOOP 2 


I? PEEK NE G 
- 30 CON.TINUE 
PLISE 


CON.TINUE® : 
é¢ RESCHEDULE THE NEXT STOP.SIMULATION 
AEDULE A STOP.SINULATION IN TIME.LISIT/s4&. ONITS 


ee STOP.SIMULATION 


Oru) 
[ew | 
Zz 


a 


OTINE IS CALLED WREN A NODE ORIGINATES AN UPDATE MESSAGE. 
NITIAL U-4SG IS SENT TO ALL OF THE INITIATING NODE*S NEIGHBORS 


DATE.MESSaGE GIVEN SENDING. NODE AND TYPE. MESSAGE 
Se-eUP.STARIS + 7% 

TO LINKS, DO 

PACH LINK TO SEE IP SENDING.NODE IS ON ONE END 


1) = SENDING. NODE OR LINK.ABLE (I,2) = SENDING. NODE) 


i<j} eee emit 22 
wR 
AA 


Hd wo 


d gz, TPDATE 
ER(MESSAGE) = SENDING.NODE 
NK.ABLZ(I,1) = SENDING. NODE 


ttgy IF 
ae 


Beg a Pr, We DEI) & Lr (ni SAGE ey NE FAMILY SE NODE) 
LED SEAN. MESSAG AMILY (SENDING.NO 
LET DESTINAT tron SES SAGE} = FAMILY (SENDING. ODE) 
GO LIST. NEXT. ST e 
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ELSE 
wes 
‘'¢g¢ IP OPPOSITE NODE IS IN ANOTHER GROUP (SAME PAMAILY) 3 


ee 
IP GROUP ete cgay) NE GROUP ee eels) 
LET GRP (MESSAGE) = GROUP (SENDIN peewee 
LET DESTINATION fee een) = GROUP (SENDING. NODE) 
ehee LIST.NEXT. STO a 


‘tee IP OPPOSITE NODE IS IN SAME BASIC GROUP 


LET DESTINATION (MESSAGE) = LINK.ABLE(I,1) 
‘LIST. S2XT. STOP! 


sree? NEXT.STOP (MESSAGE) = LINK.ABLE(I,2) 

IF FAMILY LINK.ABLE(T,1)) NE FAMILY (SENDING. NODE) 
LET PAM.LY (MESSAGE) = PAMILY (SENDING. NODE) 
LET DESTINATION (MESSAGE) = FAMILY (SENDING. NODE) 

ee INCL. NEXT. STOP 

IP GROUP (LINK. ABL2 (I, 1)) NE GROUP (SENDING.NODZ) 
L2T GRP (MESSAGE) = GROUP (SENDIN NODE) 
L®T DESTINATION fH BSSAG2) = GROUP (SENDING. NODE) 

2190 INCL-NEXT. STO 

“LEY DESTINATION (NESSAGE) = LINK.ABLE(I, 2) 

‘INCL. SZ2XT. STOP! 


LET NEXT.STOP (SESSAGE) = LINK.ABLE (I,1) 
REGARDLESS 


vere SCEEDULE ARRIVAL OF U-MSG AT OPPOSITE HODE : = 
SCHEDULE AN ARRIVAL. MESSAGE GIVEN MESSAGE IB 


O.XMN.TIME UNITS 
FZSARDLESS 
OCP 


‘ 
7 ce SCZEEDULE THE NEXT ORIGINATION OF A U-MSG FOR THIS NODE 


SCZZTILE A NEW. UPDATE.M2SSAGE GIVEN SENDING. NODE AND UPDATE A = 
oprdegns OM P (EARLIEST. UPDATE, LATEST.UPDATE, 3)) | 
SND ‘to? NEW.UPDATE 
a) rs 
''¢zg ==-S ROUTINE CREATES CONTINUED U-MSGS REPRESENTING THE RELAYING 
‘tee iF A U-HSG POM A NODE AFTER AN UPDATE 
DVENT CONT. UPDATE. MESSAGE GIVEN LAST.NODE, NEXT.NODE, NET.CY, ~ 
SOJ?5i, FA.SILY, HOP.CNT AND GR.OUP 
Gras > - MESSAGE 

2 ZEPE (HESS AGE) = UPDATE 

=> SELAYER (MESSAGE) = LAST.NODE 

“F> YEXT.STOP (MESSAGE) = NEXT.NODE 

i> HESTINATL i (4SSSAG2) = SOURCE 

"2 CHANNEL. 7ALUE (MESSAGE) = NET.CY¥ 

“2 GRP (MESSAGE) = GR.OUP 

Ble PAM any MESSAGE) = PA.MLY 
|, be= NODES.HOPPED (MESSAGE) = HOP.CNT 
tee T=IS RELAYED U-4SG IS SCHEDULED TO ARRI+va AT THE NEXT DESTINATION 
‘tge L?TER A SELECTED TRANSMISSION TIME 

SCSTIITLE AN ARRIVAL. MESSAGE GIVEN 4ESSAGR I¥ 

J.2"5. TIME ONITS 
RETII9 
ZND ''C? CONT.UPDATE 
8 
‘tee *=TS ROUTINE PROCESSES AN UPDATE MESSAGE AS IT ARRIVES IN A NODB, 
't¢z 2?LAYING IT TO NEIGHBORS IP APPROPRIATE 
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’ ¢ 
PVENT ADRIVAL.MESSAGE GIVEN ID. MESSAGE. NUMBER 
LET *2=5AGE = ID. MESSAGES. NUMBER 
LET “225.NODE = NEXT. STOP (MESSAGE) 
LET S27.U. LINKS = NET.U.LINKS + 1 
LET SD2S.HOPPED(MESSAGE) = NODES.HOPPED(MESSAGR + 1 
''ce rd C¥ OF LAST RELAYING NODE — 
FOR I = 170 6, D ' 
IP Y=EIGHBOR. LIST (THIS.NODE = RELAY ER (MESSAGE) 
22> CV.OF.LINK = NEIGHBOR. Ltst’ (THIS.NODE, I, 3) 
5) SELECT.BEST. PATH 
ELS2 
LOOP 
‘SPLOT. BEST. PATH? 
LET (OTAL.CV.OF.PATH = CV.OP.LINK + CHANNEL. VALUE (MESSAGE) 
t 
''gé ID PREVIOUSLY SELECTED BP NEIGHBOR 
LET £?.SEIGHEOR = BEST. PATH (THIS. NODE DESTINATION (MESSAGE) , 1) 
''¢¢g IP RELAYER = CURRENT 3P NEIGHBOR, UPDATE ITS CY TO THE DESTINATION 
‘tee 43D RELAY UPDATE 
IF RELLIER (MESSAGE = BP. NEIGHBOR 
Liz 22ST. PATH (THIS. YODE,DESTINATION (MESSAGE) ,2) = CHANNEL. VALUE (MESSAG2) 
I? PENT d= 3 
Sk~> 1 OUTPUT LINZ 
P2=N> 1 LINE WITH THIS. NODE, DESTINATION(MESSAGE), BP.NEIGHBOR, 
CASEEL. VALUE (MESSAGE) , CV.OF.LINK, TOTAL.CV.OF.PATH, TIME.Y ; 
NOD? ** UPDATES CV THRU SANE SP TO ®* (THRU *®) AS ¢8e+eee= Hue AT RH, SREREE SEC 
SEZ> 1 OUTPUT LINE | 
REGADILESS 
GO 3=LAY. UPDATE. TO. NEIGHBORS 
PLsz 
''ge TP THERE WAS NO BP NEIGHBOR, ADOPT RELAYER AS BP NEIGHBOR AND 
‘'ge i2LAY UPDATE 
IF 22.SZ2IGHBOR = NONE 
L=> =iST.PATH (THIS. NODE,DESTINATION N (MESSAGE), 1 = RELAYER (MESSAGE) 
Liz 257. PATH(THIS. NODE, DESTINATION (MESSAGE) ,2) = CHANNEL. VALUE (MESSAGE) 
|, bz. iFeNEIGHBOR = RELAY ER (MESSAGE) 
IP 2=4¥T >= 3 
Sx2>"1 OUTPUT LINE 
B==9" iLINE WITH THIS. NODE, DESTINATION (MESSAGE), 3BP.NEIGHBOR, TIME.Y 
772: .CV.OF.2ATH AS FOLLOWS 
Sts ZEST PATH FROM ** TO $® NOW THRU ©® AT ®B ,@emeee SEC. BEST NET CV¥= #08 
Sz=> 1 OUTPUT LINE 
REGAZILZSS 
GO EZLAY. UPDATE. TO. NEIGHBORS 
2LS2 
4 
‘tge +? THE RELAYER IS NOT THE BP NEIGHBOR, AND IP THE NEW PATH IS 
ttge “S=2ORTER THAN THE OLD BEST PATH, SAKE RELAYER THE NEW BP NEIGHBOR 
‘'ge iD RELAY THE UPDATE 
FOR > = 170 6, DO 
Tt? SE=GHEOR.Lrst(turs NODE,I,1) = 3P.NEIGHBOR 
22> CV. TO.SP.NSIGHBOR = NEIGHBOR.LIST (THIS. NODE, I, 3) 
G6) cCOMPABE.CYS 
RLS? 
Loop 
'COMNTELEZ.EVS? 
TE dee say PATS (TATS YODE,DESTINATION (MESSAGE), 2) + CV¥.T0.32. NEIGHBOR) > 
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L2T OLD.BP = BP.NZIGHBOR 
LET OLD-CV = 32ST.PATH (TAIS.NODE, DESTINATION(MESSAGE), 2) 
TPT LUNK.CY = CV.TO. 2P.NBIGHBOR 
S2T 32ST. PATH (THIS. NODE, DESTINATION (MESSAGE) , i) = RELAYER (MESSAGE) 
"3 oeeste Pate >HIS. NODE, DESTINATION(MESSAGE), 2) = CHANNEL. VALUE 
net SSAGE) non = RELAY ER (MESSAGE) 
IF PRNT d= 2 i= 
SKIP 1 OUTPUT LIN 
DRINT 1 LIVE WITH THIS. NODE, DESTINATION (MESSAGE), BP.NEIGHBOR, TIME.Y, 
CHASNEL. VALU2 (MESSAGE), CV.OP.LINK, TOTAL.CV.OF. PATH As FOLLO@S 
WEd BEST CATH FROZ ** TO NOW THEU ** AT Se, Semeeee SEC, CV= coeeees bee 
DRIN- 1 LINE wITH OLD. 32 Orome ve LXKC¥, (OLO.cV + LNE- CV¥) AS POLLOWS 
OLD BP THRU =* HAD CY xin > amge’s 838 
SXID 7 OUTPUT LINE 
REGARDLESS 
»1 89 RZLAY.UEDATE. TO. NEIGHBORS | 
oc IP S2W PATH IS NOT BETTER THAN THE OLD BEST PATH, DISCONTINUE U-4SG 


2 DISCONTINUL.ORIGINAL.MESSAGE 


*‘tge IP A NEW BP IS SELECTED OR AN OLD 3 
*‘tez FOR THE NEXT U-4SG TO ALL NEIGHBOR 


| ZELAY -OPDAT!.T0.NEIGHBORS* 


T (THIS. WODE = BP. NEIGHBOR 
NE1G55OB = nErangor. LIST (THIS.NODB, I,- 3) : 


2 IS UPDATED, PREPARE INFORMATION 


.NODE = BEST. PATH (THIS.NODE, DESTINATICN (MESSAGE) ,2) 


4zwoo 
zie 


Ey Ty : YES 
DE N? RELAYER (MZSSAGE) ~ 
gBoR bre (THIS.NODE, I, 1) 


= 
E ES IN ANOTHER FACZLY AND THES IS A INTHA-FANILY 
G 
i 


nr 

we 

ary 
' 


~ me 


MW KQ Ghine 


we Wig mQay 
mr AM NmMor 
bi ar 


Sea O AND FPAMILY(UPSTREAH.NODE) NE FPAM.LY (MESSAGE) ) 


a id 
tut¢ 


a NODZ IS IN ANOTHER GROUP AND THIS IS A INTRA-GRCOP (SABE 
Ssayech Lae LT 


SAGE) NZ 0 AND GROUP (UPSTREAM. NODE) NE GRP (MESSAGE) ) 
S.-UPSTREAM 


Be coeur Loe Toe SARE BASIC GROUP AND THIS IS A BASIC ° 
OZIGINATED U-MSG, 2ELAY IT 


UF(UPSTREAN.NODE) = SROUP (THIS.NODE) 
sf ES CY el aa GRP (SESS AGE) ) 
TEANS.OUPS?2as 


E ABOVE, UPSTREAM NODE DOES NOT GET A U-4SG 


Cr = 


rarg 
3 


IVE¥ THIS.NODE, UPSTREAS.NODE, 
ION eee a Bath PAM. LY (MESSAGE) , HOPS, 
SeiuG : AC.RATIO) UNITS 


oon Pej 

CTV) tm oh Lope 

ro 

MEO amew 

Me OORHR 
Ve ood OS > - © 8 
Qatejin 
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© DESTROYED AFTER TRAVELLING INE 
FROCEEDS ACCORDING [TO THE 5ASIC 


E SIMOLATION, ALL ue 
Ec 
Sena es CAUSES NEW U-MSGS TO BE 
iC O 
4S 


J- 
HOWEVER THE UPDAT 
TeSBEGCAUSE THE ARRIL 
oe TED OR AN OLD BP WAS UPDATED. 

N 


EDS he eA NEw) BP 
OZING U-aSG, IT IS DESTROYED 


DE COULD NOT US 
GENERATING ANY 
Q 


GINAL. NESSAGE® 

MESSAGE} > MAX.0.HOPS 
NODES. HOPPED (MESS AGE) 
SAGE CALLED ID.MESSAGE. NUMBER 


RIVAL. UPDATE 


@ 

R 

a 
HHO wa 


4 


Ui 4 
il 


igi 4 rigor 
DSU bse fel 
mule rw crete ler ts. 


LATES CHANNEL VaLUES BASED ON A TIME-WEIGHTED 
SIZES OVER A SPECIFIED TIME CALLED THE WINDOW. 
ATION OLDER TH4N THE WINDOW TIME IS DISCARDED. 


0 
4 


gg THIS ROUTINE 


ENE. CALC 
VERAGE OF QUEUE 
ZE INFOR 


Pach Me LAS 90 SPAN, MID, AREA, WEIGHT, BLOCK 
aEAL VARIABLES 


T 
S 

= 1 T0 N.NODE, DO 
DE0E NFORMATION BEYOND WINDOW SIZE. "PACK" IS A PACKAGE 


RR WHsVwdrd AR 


RA 
Oo 


er 
ION DESCRIBED LATER. 

ris E-QUEUE (THIS.NODE) WITH ENTRY.TIME(PACK) < 
M’TIMPB.QUBUE (THIS. NODE) 


tyes? ee2e'ht Uig2* *e@ @ eigagtiwhw 
Oe ec « eOrmrtdd «© 2 ewe eotinthty ty 


"tg¢ CALCULATE THE CY TO ZACH NEIGHBOR 
FOR J | 
I 


i.) | 
ta 
tseyg 

tH 
GY ae 


R.LIST (THIS. NODE, J, 2) = YES 
NEIGHBOR.LIST (THL£S.NODE, J, 1) 


ty 
e 
< 


ug 
Oo moO m0 


‘UO 
© 


M1 ‘Cts = 


42Owstyity - 
ber oe Cl bet 


CCMA OR Am OMA tt 
as 
rO 


-QUEUE (THIS.NODE) WITH PAC. NEIGHBOR (PACK) = 


td Urjrzrgrgr] 


H+ 
Witt ny 
bs] 

im ONAN? HO tae 


a A AM Ore 


ii 
WrecgaQ potty 


LET 
Y.TIME (PACK) 


ER (PACK) ) " SPAF 


nN 


af 
DG 
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4O Koni eet Hille 
wt A as 


N 
fe ACK) 
MBER (PACK) ) 


tie* 


Mintstghiytd dO 
HHHANANHOr: 


4 
or] 


NITOR (THIS.NODE, YEIB, 2) 


NHN WING KOI e a O 
mine 


oe betgbety Bebe Uditlittunoule « 
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te hte eit 
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LET NEIGHBOR.LIST (THIS.NODE, J, 3) = CV¥.OP.LINK 


REGARDLESS 
oop : 
L90P 
**¢¢ SCHEDULE THE NEXT CV CALCULATION FOR ALL NEIGHBORS. IN THE 
*te¢ SIMULATION, THIS PROCESS IS SYNCHRONIZED POR EVERY NODE L¥ 
tteg THE NETWORK (SEE CHAP VI). 
SCHEDULE A CV.LATCH IN (2*UP.DATE.PERIOD) ONITS 
‘¢¢¢ PARLIEST.UPDAT2 AND LATEST.UPDATE SAT THE NEXT INTERVAL DURING 
‘tee WHICH ALL NODES WILL PANDOMLY INITIATE AN UPDATE CYCLE. THE 
eee NEXT CV.LATCH FOR ALL NODES IN THE NETWORK OCCURS AT THE VERY 
‘tee SIGINNING OF THIS INTERVAL, AFTER THIS PERIOD, THERE IS ANOTHER 
*'e¢g FOURL SIZED PERIOD DURING WHICH NO UPDATE CYCLES ARE INITIATED. 
‘tge SUT THIS PERIOD INSURES THAT ALL CYCLES STARTED DURING THE 
‘ge FARLIEST.UPDATE TO LATEST.UPDATE PERIOD WILL BE COMPLETED. 
‘'ge DURING THESE TWO PERIODS, THE CV FOR ALL LINKS ARE FROZEN. 
LET FASLIEST.UPDATE = TIME.V + (2% UP.DATE. PERIOD) : 
Lin LATEST. UPDATE = TIME.V + (3” UP. DATE. PERIOD) 
2ND 'tOP CY.LATCH 
| | 
‘tg¢ THIS ROUTINE GENERATES A TRAPFIC S2SSION MADE UP OF A RANDOM 
**¢¢ N“OMBER OF PXTS (BETWEEN PRESCRIBED LIMITS). PKTS ARE SENT OUT 
"'¢¢ ON IDLZ LINKS Iz AVAILABLE, OR STORED IN QUEUES IF LINKS ARE BUSY. 
SVENT N2W. PACKET. MESSAGE GIVEN T.MESSAGE 
SEPINZ CK.XMTR, CK.RCVR, X. TOT. PERCENT AND R.TOT.PERCENT AS REAL 
VARTABLES 
LI? X.TOT.PERCENT = 0 
LET P.TOT. PERCENT = 0 
RET CK.XNTR = UNIPORM.P(0.0, TRNS.PCNT, 2) 
*'¢¢ SELECTOR IS USED IF A PERCENTAGE OF THE TRAPPIC IS REQUIRED 
*'¢¢ TO SE INNER-GROUP/PAMILY 
T2T SZLECTOR = ONIPORM.P(0.0, 100., 9) 
‘'¢e SELECT THE TRANSMITTING NODE —., 
70R I = 1 TO N.NODE, DO 
LET X.TOT. PERCENT = X.TOT.PERCENT + TRANSMIT. PERCENT (I) 
IP CR.XETR <= X. TOT. PERCENT 
LE? XMTR = I 
GO PIND.RECSIVER 
2LS2 
790 
*'¢¢ SELECT TBE RECSIVER 
* SIND. RECEIVER? 
=> CK.RCVR = UNIPORM.F (0.0, RCV.PCNT, 3) 
>O2 J = 1 TO N.NODE, DO 
Pet 2.T0T.PERCENT’ = R.TOT.PERCENT + RECEIVE. PERCENT (J) 
IP CK.RCVR <= R. TOT. PERCENT 
LET R8CVR = J 
GO CK.GROUPS.AND. FAMILIES 
SLS2 
£902 
‘'¢¢g IP THE RECEIVER MUST 32 INNER-GROUP OR PAMILY, KEEP LOOKING UNTIL 
‘'g¢ AN ADEQUATE RECEIVER Is FOUND 
*CX.GROUPS.AND. PAMILIES® : 
I> SELZcTOR < IN.GROUP 
ne S209? (XATR) | . G200P(RCVR) 
GO SEE. IP.XMTR.EQ. RCYR 


ge) 
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SE 
LET R.TOT.PERCENT = 0.0 
GO PIND. RECEIVER 
ZLSE 
Er SELZCTO “GROUP + IN-PAMILY) 
iP PANILY (29 a) =" PAMILY (RCVR) 
GO SSE. LF.XMTR.£Q.RCVR - 
ELS2 | 
L2T R.TOT.PERCENT = 0.0 
GO PIND. RECEIVER 
ELSE 
*SEZ.IP .XATR.2Q.RCWR® . 
TP RCVR = XAT 
LET 2. TOT. BERCENT = 0.0 
GO PIND. RECEIVER 
ELSE 
* DERIVE. NUMBER.OP. PACKETS? 
LET PAT.COUNT = INT.F(UNIFORM.P(PKT.MIN, PKT.MAX, 4)) 
''e COUNT TOTAL TRAPPIC SESSIONS. THE TRACER ARRAY KEEPS TRACK OF 
‘tge SESSION INFORMATION 
LET TOT.NEW.TRAPPIC = TOT.NEW.TRAPPIC ¢ 1 
LZT TRACER {TOT.NEW. TRAFFIC, 1) = PKT.COUNT 
''¢¢ CREATE A MESSAGE FOR EACH PACKET 
POR I = 1 TO PKT.COUNT, DO 
CREATE A MESSAGE 
Lz TEP! (MESSAGE) = PACKET 
Lit RELAYER (MESSAGE) = XMTR = - 
feeesz PC™= 0 : 
‘'¢¢ ADDRESS PRT TO NODE, GROUP OR PAMILY OP DESTINATION AS APPROPRIATE 
IP FAMILY(XMTR) NE SRE SER 
L2T BP.NODE = BEST.PATH (XMT gABILT (CVD) » FE 
Let £8.GP (MESSAGE) = FAMILY ( 
- Eat PG = FAMILY (RCTR 
30 ADD.DESTINATION z 
ELSE 
I> GROUP (XMTR) NZ_GROUP PRCT RY 
Let 32-SODE 2 S287. PATH (XNTR, GROUP(RCTR). 1) 
L2t 24.GP (MESSAGE) = GRO RCV R = 
LET FG = GROUP (RCTR) 
GO ADD.DESTINATION 
ELSE 
LET BP.NODE = BEST. PATH (XMTR, RCVR, 1) 
* ADD. DESTINATION® E 
LET DESTINATION(NESSAGE) = RBCYR 
E22 TRANS. NUMBER (MESSAGE) = TOT. NEW. TRAPPIC 
L27 pACK. NUMBER (MESSAGE) = I 
_, bit SEXT.STOP(MESSAGZ) = BP.NODE 
"'¢¢e IF LINK TO BP NEIGHBOR IS IDLZ, SEND OUT PACKET 


IP LNX.MONITOR (XMTR, BP.NODE = IDLE 
SCHEDILE AN A IVE. SACKET betsy SESSAGE IN PKT.XMN.TIME UNITS” 
LNK -¥ONTTOR (XMTR, 3B.NODE, 1) = 

2? RELEASE.TIME (MESSAGZ) = TIM 


2 ae IP LINK IS BUSY, STORE OKT AT END OF QUEUE FOE THAT LIEK 


FILE ZESSAGE IN QUEUE (XMTR 
LE? LNK  NONITOR (XMTR, SP. NODE = LNK.MONITOR (XUTR, 32.NODE, 2) + 1 
IP LNK. MONITOR (XMTR,32-. NOD = 5f >i! Se UG een OEE 3 
Lit LYK. HONTTOR (XATR,3P.NODE,3) = LNK.MONTTOR (XATR,3P.9ODZ,2) 
_, REGARDLESS 
‘tec IP LINK QUEUE CHANGES, CREATS A "PACK" (A PACKAGE OP INFORMATION}= 
"*¢¢ HICH CONTAINS THE NEW QUEUZ SIZZ AND THE TIME IT #AS CHANGED 
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2 ee POR THE LATE CALCULATION OF CE 


CREATE 4 PACK 
LT NUMSER(PACK) = LNK.MONTTOR (XSTR,3P.3ODE,2E: 
LET ENTRY. ISS (PACK) = TIMB.V 
LET PAC.NELGIS0R (PACK) = BP. NODE: 
FILE PACK IN TIME. QOEUS (XINTR}: 
_ REGARDLESS , 
TF PRNT >= 1 
PRINT 1 LINE wWITZ TOT.NES. TRAFFIC, I, XMTR, BP-NODE,2G, BCVR, TINE-¥ 
LNX.MONITOR(XMNTZ, BP.NOD2, 2) PCLLO 
ewnaar see TNITIATID FROM 2% THAD sn iam) “5 a AT 8, S@eee" SEC. QU= *2e- 
SKIP 1 OUTPUT LINE 
D>FGARDLESS 
LOOP 
¢ 8 ~ 


‘ice RESCHEDULE NexT TRAPYIC SESSION UP TO SAX SeT BY TRAPPIC.LISIT:: 


oe SOLs Nowe LRAEF ILC < TRAP. LISIT 
SCHEDULE A NEW.PACKET.MESSAGE GIVEN PACKET IN: 
EXPONENTIAL. P (AVE. NEW. TRAPPIC. INTERVAL, 1} UNITS. . a 


PEGARDLSSS 
RETORN 
2) WUC: NEW. PACKES- 
¢9 
‘'¢¢ THIS ROUTINE PROCESSES. PRTS AS THEY ARRIVE INA NODES 
eT AT VE- PACKET GIVEN ID. 3U MBER: 
fat SO 8 = oe ae 
Hit TATS.NODE = VEET. STOP (TESS AGEF:. - 
LET PAST. YODE = a2ZLa7z Za (MESSAGE). 
IF PRNT >= 1 
SINT 1 LINZ wre TRANS. SESE ERSTE PACK. NUMBER (MESSAGZF ge 
ZELAYER (SESSAGE) , NEXT .STO? (4ZSSAGE) PIME. AS POLLOWS- 
scr y= ARRIVES P20" ** Iyrd =* ar _aeeake SEc 
ZEGARDL2SS 
‘'ce IP THE PRT HAS R2ACHED ITS DESTINATION, GO TO A PROCESSING a00TINE= 
I? NEXT.STOP(MESSAGZ) = DESTINATION (MESSAGE) 
|, SCHEDILE A COMPLETED. TRIP GIVEN MESSAGE ¥EXT™ - 
IF PRNT >= 1 
PRINT 1 LINE AS FOLLOWS 
AKD STOPS 
2 ZGARDLZSS 
'e¢¢ ITP OKT IS TO CONTINUE, ADDRESS IT TO THE NEXT BP NEIGHBOR BASED- 
‘tee ON THE NODE, GROUP OR PAMILY ID OF THE DESTINATION aS APPROPRIATE 
ELSE 
LET S2LAYZER(MESSZGE) = THIS. NODE. 
Let oa-GP (Ms SSAGZ) = 0 
LG a» SS = 
IF PASILY (THIS.TODE) NE PA&MILY (DESTINATION (MESSAGE) F 
Let 7G = SANTEE (DESTINATION (MESSAGE) ) 
aati, ree ry 
L3™ P¥.GP {22S3852) = ?¢ 
2192 ASGN.NEXT.STOP 
I? GROUP (THIS.NODE) NE seoopP eee: (MESSAGE) ) 
cal ae = jgR0UE (DEST TINATION (HESSAG2) ) 
L2T F™.G? {#283852) = 76 
=138 ASGN.NEXT.S7OP 
~ LET §D.O0BJ = DESTINATION (MESSAGE}- 
*ASGN.NEXT.STOB®* 
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LET NEXT.STOP (MESSAGZ) = BEST.PATH (THIS.NODE, BP.OBJ,1) 
LET NODES.HOPPED (MESSAGE) = NODES. HOPPED (MESSAGE) + 1 
L 
‘'¢¢ SCHEDULE A P2OCZSSING COMPLETION TIME WHEN THE PKT WILL BE READY 
‘tge FOR RETRANSSISSION. 
SCHEDULE A CON.PACKET.MESSAGE GIVEN MESSAGE IN PROCESSING.TIME UB8ITS 
PEGARDLESS | 
‘'¢¢ GO TO THE QUEUZ OF THE NODE WHICH RELAYED THZ ABOVE PXT. | IF 2MPTY 
‘'¢¢ DEPINE THE LINK AS. IDLE: IP NOT, PLACE THE NEXT PKT ON THE LINK 
‘tee AND ADJUST THE QUEUE INFORMATION BY CREATING A NEW PACK. 
POR BACH MESSAGE IN QUEUE(PAST.NODE) WITH NEXT. STOP (MESSAGE) =THIS. NODE, 
FIND THE PIRST CASE 
I? NONE 
eres? LNK.MONITOR ( PAST.NODE, THIS.NODE, 1) = IDLZ 
& rw 
RENOVE MESSAGE PROM QUEUE (PAST . NODE) 
LET LNK. MONITOR (PAST.NODE, THIS.NODE, 2) = 
ENK. MONITOR (PA T.NODE, THIS.NODE, 2f = 1 
CREATE A PAC 
Let NUMB=R (PACK = LNK.MONITOR (PAST.NODE, THIS.NODE, 2) 
L2t ENTRY. TIME (PACK) = TIME. V 
LET PAC.NSIGHB R( EA K) = THIS.NODE 
PILE PACK I TIME.QUZUE (PAST. NODE) 
IP NODES. HOPPED (MESSAGE) = 0 
LET RELEASE.TIME (MESSAGE) = TIME. 


a REGARDLESS 
ee SCHEDULE THe ARRIVAL OF THE PKT JUST RELEASED PROH THE QUEUEW 
: SCHEDULE AN ARIVZ.PACKET GIVEN MESSAGE IN PKT.XMN.TIME UNITS 
F PENT d= 
PRINC 
RELA 
(MES 
SRRRES sR LE 
SEGARDLESS 
5 ge Id - 


UR! 
**OP ARIVE.PACKET 


4 


R (MESSAGE) , PACK. NUMBER (MESSAGE) ¢ 
MESSAGE), 24.GP (MESSAGE) , DESTINATION. 
a (PAST NODE, PHIS. NODE, 2) AS FPOLLOSS 
®) AT 2” enkame SEC (F204 Q) QU= see 


A tL tt 

Re Oy. 

Rresuitg 
on ed) 
pe o* 2; 


1 
Tee 
SAG 
AVE 


Or 


“wees 


me otto ROUTENE CONTINUES THE PACKET ON 7 BEST PATH APTER SROCESS HG 
Porte ths GINK {5S EDLE, OR PLACES IT IN a2 QUEUE. 


NT CON.PACKET.MESSAGE GIVEN IDENT.MESSAGE. NUMBER 

EZSSAGE = ITDENT.A4 ESSAGE. NUMBER 

THIS.NODE = RELAYER (MESSAGE) 
€ IF LINK IS AVAILABLE, SEND OUT PKT. LIST LINK AS BUST.~. 


Zeus. ONLTOR THIS.NODE, PICNIC LC ana oce ut = IDLE 
pp De HZDULE AN ARIVE.PACKET GIVEN NESSAGE IN PKT.ZAN.TISE UFITS : 


2 2 etjpj< «oe © @ etry 


ig@e32e wmttritu ar aee ating 


SAGE), PACK.NUMBER (MESSAGE) , 
GE) Pu GP (MESSAGE) , 
OL ba 


S 
AT @# ,S8e8O82 SEC. NO WAITH. 
ee LNUK.MONITOR(THIS.NGDE, NEXT.STOP(MESSAGE), 1) = BUST 
vr 


''¢g¢ IP LINK WAS 3USY, PLACE PKT IN QUEUE AND CREATE A PACK @ITH SEF 
''e¢ QUEUE INPORMATI ON. 


FILZ MESSAGE IN QUEUE (THIS. NODE) 


Hirsi (e 
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EXT.STOP(MESSAGE),2) # 


L2? LNK.MONITOR (THIS.NODE X 
N 37 P(MESSAGE), 2) ¢ 

5 

TE 

ere 


LNK.MONITOR (THIS. NODE 
IF LNK.MONITOR (THIS. NODE, 
LNK.MONITOR (THIS.NODE, 
MONITOR (THIS .NC 
TOR (THIS.NODE, 


TO 1 
STOP HESS AGE) 2) > 
TOP (MESS AG k| 

x Eg 


Owes 


EXL.STOP 


N 
T 
5 
T 
be AGE}, 3) = 
& TOP (ME SAGE), f 


e 
EX 
N 
NEX 
DE 
N 


. 


Ow 
wid 
tr Gy 
st ee otis 
2 ach abel ae 
wor 
inz Eve 
Soe ty ka 


race uonr 


I 
ACK é : 
ER (PACK) = LNK. 40 NONTTOR (THIS. NODE, NEXT.STOP(SESSAGE), 2) 


E2r PAC NEIGHBOR (PACK) = = ES. - STOP (SESS AGE) 
FILZ PACK IN TIME.QUEUES (THIS. NODE) 


IF O8NT >= 


t 


N 
S 
P 
3 
R 


eo acne Gee PACK.NUMBER ( MESSAGE 
eit TIME.V, LNK. MONITOR (THIS. 30D 
5 eee 


AT he tHe SEC, QU = ae 


OH Ot 


THIS ROUTINE COLLECTS STATISTICAL DATA WHEN A PKT REACHES ITS 
DESTINATION. 


COMPLETED.TRIP GIVEN MES.N 
2 DEL. TIME A REAL VARIAB 
2gSAGE = NES_MUM 
WOR = NODES.HOPPED (MESSAGE) + 1 - 


¢ PRINT ALERT IF NODES HOPPED >= TOTAL NODES IN NETWORK. 


FP (CNTR >= _N.NODE AND MSG.HLT = 0) 
PRINT 1 LINE AS FOLLOWS 

PROBLEM -- MORE HOPS THAN NODES 
LET ¥SG.HLT = 1 

GARDLZSS 


INCREMENT COONTER FOR TOTAL NODES dOPPED FOR THIS PKT AND SUA NEP 
TISE POR GIVEN NUMBER OF dOPS. 


7FOP.COUNT (CNTs PACKET) = dOP.COUNT iene PACKET) ¢ 1 Fog e 
SL.TIME = TIME.Y - RELEASE.TIME (LZs E) 

CLOCK.DATA (CNTR, 1) = CLOCK.DATA (CNTR, 1 )* DEL.TIME 

EevOte 1 THIS TRANSIT TIE 15 A NEW MEX FOR TEIS NUMBER OF HOPS. 


> CLOCK.DATA ee on 
DATA (CNTR, 2) EL. TINE 


U 
L 


--© stunting 2 2 © © etytd <7 
aR WHIM AR 


HH * © ett*r+UYy © © © @ atyiy ey % 


t4 = @ aptty 4s «2 2 ad 

hy oe © efijtjid «© « « ae 
RB HH AR 
aA 


Nz 
CK. 
S 

EGCNT TRACER ARRAY WHICH KEEPS THE NUMBER OP PKTS GENERATED 
THE NOMSER REACHING THEIR DESTINATION. 


R_ (TRANS.NUMBER(MESSAGE), 2)= TRACER (TRANS.NUMBER (MESSAGE) , 2) +1 
BSSAGE CALLED MES. NUS 


FP COMPLETED. TRIP 


DPeGtuks OVES THE FIRST 


Soa OF THE S 
E SAMPLED DURING THE 


A BU 
SO THE LINAS Ca 


RAR 


aj *® 2 2 @ @ ait} yr a a2 @ a'D 
are eoeeee ea@Zirt) «© « @ aly 
RAR 


ENT QU.SAMPLER 


PRINT 1 LINE AS FOLLOWS 
32-2 -- === $= = == 2 + = 5 +--+ == - = += MID POINT 


{4 
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nee ID THE LARGEST QUEUE SIZE IN THE FIRST HALP OF THE SIMULATION. 


LET I= 1 
POR P = 1 TO N.NODE, DO 
FOR T = 1 TO N.NODE 
IP MAX < LNK: 40Nt FOR (7,7 3) 
L2T MAX = LNK.MONITOR (P,7,3 
REGARDLESS 
LOOP = 
LOOP 
''z¢ SHP.LINKS IS AN INPUT VARIABLE LISTING THE NUMBER OP LINKS TO 3B 
‘tg¢ SAMPLED POR QUEUE SIZE. 
‘tee S4P.SET IS AN ARRAY CP NODE PAIRS FOR THE "SMP.LINKS"™ BUSIEST® 
‘'g¢ LINKS OVER THE FIRST HALF OF THE SIMULATION. 
‘PILL.SNP.S2T? 
POR F = 1TO N.NODB, DO 
POR T = 1 TO Y.NODE, DO 
IP LYK.MONITOR (BoE e3) MAX 
L?T SMP.S=T zy 3) = 
L27 SMP.S2T(I,2) =f - 
If I = SMD.LINKS 
GO -BEGIN. SASPLING 
BLS2 
L2T © = Ie 
Be aco HOP. NEW. SAX 
I? N2w. MAX < LNKX. MONITOR (P ete2k 3 AND LNK.MONITOR(P,T,3) < SAX 
L2t NEW.MAX = LNK.MONITOR(P, 
REGARDLESS 
‘HOP. SEM@.SAX® - 
LOOP 
LOOP 
L2? MAX = NEW.MAX 7 
GO PILL. SMP. SET 
‘'¢¢@ SCHEDULE FIRST SAMPLE OF ALL LINKS IN SMP.SET 
‘2 EGIN.SAMPLING® ; 
L=™ TILMER = TIN2.LIMNIT TZ (2% REAL.P NQ. OF SAMPLES) ) 
S i EDULE hk SAMPL2 IN (EXPONENTIAL. P(II.4ER, 38)) UNITS 
D'* OP QU.SAMPLZB 7 


se 


2 THIS ROOTING SAMPLES THE LINKS IDENTIPIED IN QU.SAMPLER WITH A 
Z€ EXPONENTIAL SAMPLING RATE. 


ENT SAMPLZ 
gé COONT ACTUAL SAMPLES TAKEN. 
T SHP.CNTR = SMP.CNTR + 1 


Of FAW seg * 6 Seb a eOiO ny 


‘gy ee eri ee alY @ e222 @ aig 


¢¢ INC2Z2NENT COUNTER BASED ON QUEUE SIZE. 
R I= 170 SMP.LINKS 
LET OU.DISTR LYK, #37208 (sup.szt T{T, 1) S825 82 S27 | 12, 2) +1), 2 
QU.DISTR (LNK.4ONITOR (S“4P.SET(L P.SE re yet 
,00P 
(tz¢ SCHEDULE THE NEXT SAMPLE. 
ieces 423 1S DEFINED IN 2U,SAMPLER 
SCHEDULE A SAMPLE IN (EXPONENTIAL. F(TI.AER,8)}) UNITS 
2 >TURN 
=ND ''OP SAMPLE 
7/SIMU06 DD SYSOUT= A 
//7*0.SI4006 DD VYOL=SER=MVS004 ,UNIT=3350, DIS P=SHR, DSN=S2034.R0N, 
77* DCB= (EECPM=FS, LRECL=133, BLKSIZE=4 123) 


iSite 
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D 
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PILZ: 


INPUT FILE <-- 


--> TYPICAL 


fxq 
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ORT +3 MUO a 
Hui. O«agerm 
MLW LHmMmMaAgOEX 
pa eee HUE HO 
MU om by oz? BNO 
Be aS C0009 AHHMNM= 
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NOMBER OF SAMPLES EACH 


hag 
=| 
tee 
wi ee 
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fm - bf 
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s 
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OUTPUT EXAMPLE 
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MEAN NUMBER CF NODES HCPPED PER PACKET IS 
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